Е-Публикации
Технически университет - София

Abstract: Copyright In the cultural heritage domain, writer recognition has become a challenging classification task still explored for historical documents, due to the presence of different types of noise in the documents, i.e., ink bleed-through, ink corrosion, stains on paper or parchment, difficulty in the character discrimination, elements different from the text, such as images, etc. that limit the effectiveness of existing techniques. To further advance in terms of robustness of classification and experimental setting, we propose a new deep learning model which ensembles pre-trained convolutional neural networks for writer recognition. Specifically, the ensemble is composed of three pre-trained Inception-ResNet-v2 models with different hyperparameter values. Results obtained on the benchmark ICDAR 2019 dataset of handwritten historical documents prove that the proposed approach is very promising in recognising the handwritten characters of different writers, also when compared with other deep learning models.

References

1. Abdeljalil, G., Djeddi, C., Siddiqi, I. and Al-Maadeed, S. (2018) ‘Writer identification on historical documents using oriented basic image features’, in 16th International Conference on Frontiers in Handwriting Recognition (ICFHR), IEEE, Niagara Falls, NY, USA, pp.369–373, DOI: 10.1109/ICFHR-2018.2018.00071.
2. Adak, C., Jaswanth, B., Akhtar, Z., Kåsen, A. and Chanda, S. (2023) ‘Writer identification from Nordic historical manuscripts using transformer networks’, IEEE International Joint Conference on Biometrics, Ljubljana, Slovenia, pp.1–9, DOI: 10.1109/IJCB57857.2023.10448665.
3. Alkhawaldeh, R.S., Alawida, M., Alshdaifat, N.F.F., Alma’aitah, W.Z. and Almasri, A. (2022) ‘Ensemble deep transfer learning model for Arabic (Indian) handwritten digit recognition’, Neural Comput. Appl., Vol. 34, No. 1, pp.705–719, DOI: 10.1007/s00521-021-06423-7.
4. Amelio, A. and Pizzuti, C. (2014a) ‘An evolutionary approach for image segmentation’, Evolutionary Computation, Vol. 22, No. 4, pp.525–557, DOI: 10.1162/EVCO a 00115.
5. Amelio, A. and Pizzuti, C. (2014b) ‘A new evolutionary-based clustering framework for image databases’, in 6th International Conference on Image and Signal Processing (ICISP), Lecture Notes in Computer Science, Springer, Cham, Cherbourg, France, 30 June–2 July, Vol. 8509, pp.322–331, DOI: 10.1007/978-3-319-07998-1 37.
6. Babić, R.J., Amelio, A. and Draganov, I.R. (2022) ‘Writer identification from historical documents using ensemble deep learning transfer models’, in 21st International Symposium INFOTEH-JAHORINA (INFOTEH), IEEE, Jahorina, Bosnia and Herzegovina, 16–18 March, pp.1–5, DOI: 10.1109/INFOTEH53737.2022.9751301.
7. Bennour, A., Boudraa, M., Siddiqi, I. et al. (2024) ‘A deep learning framework for historical manuscripts writer identification using data-driven features’, Multimed Tools Appl., DOI: 10.1007/s11042-024-18187-y.
8. Boateng, E., Otoo, J. and Abaye, D. (2020) ‘Basic tenets of classification algorithms K-nearest-neighbor, support vector machine, random forest and neural network: a review’, Journal of Data Analysis and Information Processing, Vol. 8, pp.341–357, DOI: 10.4236/jdaip.2020.84020.
9. Brodić, D., Amelio, A. and Milivojević, Z.N. (2015) ‘Characterization and distinction between closely related South Slavic languages on the example of Serbian and Croatian’, in 16th International Conference on Computer Analysis of Images and Patterns (CAIP), Lecture Notes in Computer Science, Springer, Cham, Valletta, Malta, 2–4 September, Vol. 9256, pp.654–666, DOI: 10.1007/978-3-319-23192-1 55.
10. Bulacu, M. and Schomaker, L. (2007) ‘Text-independent writer identification and verification using textural and allographic features’, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 29, No. 4, pp.701–717, DOI: 10.1109/TPAMI.2007.1009.
11. Cauteruccio, F., Stamile, C., Terracina, G., Ursino, D. and Sappey-Marinier, D. (2016) ‘An automated string-based approach to extracting and characterizing White Matter fiber-bundles’, Computers in Biology and Medicine, Vol. 77, pp.64–75, DOI: 10.1016/j.compbiomed.2016.07.015.
12. Chahi, A., El Merabet, Y., Ruichek, Y. et al. (2024) ‘WriterINet: a multi-path deep CNN for offline text-independent writer identification’, IJDAR, Vol. 26, pp.89–107, DOI: 10.1007/s10032-022-00418-3.
13. Chammas, M., Makhoul, A. and Demerjian, J. (2020) ‘Writer identification for historical handwritten documents using a single feature extraction method’, in 19th IEEE International Conference on Machine Learning and Applications (ICMLA), IEEE, Miami, FL, USA, pp.1–6, DOI: 10.1109/ICMLA51294.2020.00010.
14. Chen, S., Wang, Y., Lin, C.T., Ding, W. and Cao, Z. (2019) ‘Semi-supervised feature learning for improving writer identification’, Information Sciences, Vol. 482, pp.156–170, DOI: 10.1016/j.ins.2019.01.024.
15. Christlein, V., Gropp, M., Fiel, S. and Maier, A.K. (2017) ‘Unsupervised feature learning for writer identification and writer retrieval’, CoRR, DOI: abs/1705.09369.
16. Christlein, V., Nicolaou, A., Seuret, M., Stutzmann, D. and Maier, A. (2019) ‘ICDAR 2019 competition on image retrieval for historical handwritten documents’, CoRR, DOI: abs/1912.03713.
17. Cilia, N.D., De Stefano, C., Fontanella, F., Marrocco, C., Molinara, M. and Scotto Di Freca, A. (2019) ‘A two-step system based on deep transfer learning for writer identification in Medieval books’, in In 18th International Conference on Computer Analysis of Images and Patterns (CAIP), Proceedings, Part II, Springer-Verlag, Berlin, Heidelberg, Salerno, Italy, 3–5 September, pp.305–316, DOI: 10.1007/978-3-030-29891-3 27.
18. Cilia, N.D., De Stefano, C., Fontanella, F., Marrocco, C., Molinara, M. and Freca, A.S.d. (2020a) ‘An experimental comparison between deep learning and classical machine learning approaches for writer identification in Medieval documents’, Journal of Imaging, Vol. 6, No. 9, DOI: 10.3390/jimaging6090089.
19. Cilia, N.D., De Stefano, C., Fontanella, F., Marrocco, C., Molinara, M. and Scotto Di Freca, A. (2020b) ‘An end-to-end deep learning system for medieval writer identification’, Pattern Recognition Letters, Vol. 129, pp.137–143, DOI: 10.1016/j.patrec.2019.11.025.
20. El-Metwally, S., Zakaria, M. and Hamza, T. (2016) ‘LightAssembler: fast and memory-efficient assembly algorithm for high-throughput sequencing reads’, Bioinformatics, Vol. 32, No. 21, pp.3215–3223, DOI: 10.1093/bioinformatics/btw470.
21. Fiel, S., Kleber, F., Diem, M., Christlein, V., Louloudis, G., Nikos, S. et al. (2017) ‘ICDAR2017 competition on historical document writer identification (historical-WI)’, in 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), IEEE, Kyoto, Japan, Vol. 1, pp.1377–1382, DOI: 10.1109/ICDAR.2017.225.
22. Garz, A., Würsch, M., Fischer, A. and Ingold, R. (2016) ‘Simple and fast geometrical descriptors for writer identification’, in IS&T International Symposium on Electronic Imaging 2016, Document Recognition and Retrieval XXIII, IS&T Library, San Francisco, California, USA, 14–18 February, pp.1–12.
23. Gattal, A., Djeddi, C., Abbas, F., Siddiqi, I. and Bouderah, B. (2023) ‘A new method for writer identification based on historical documents’, Journal of Intelligent Systems, Vol. 32, No. 1, p.20220244, DOI: 10.1515/jisys-2022-0244.
24. Hem, S. and Schomaker, L. (2016) ‘Co-occurrence features for writer identification’, in 15th International Conference on Frontiers in Handwriting Recognition (ICFHR), IEEE, Shenzhen, China, pp.78-83, DOI: 10.1109/ICFHR.2016.0027.
25. He, S. and Schomaker, L. (2017) ‘Writer identification using curvature-free features’, Pattern Recognition, Vol. 63, pp.451–464, DOI: 10.1016/j.patcog.2016.09.044.
26. He, S. and Schomaker, L. (2020) ‘FragNet: writer identification using deep fragment networks’, IEEE Transactions on Information Forensics and Security, Vol. 15, pp.3013–3022, DOI: 10.1109/TIFS.2020.2981236.
27. He, S. and Schomaker, L. (2021) ‘GR-RNN: global-context residual recurrent neural networks for writer identification’, Pattern Recognition, Vol. 117, p.107975, DOI: 10.1016/j.patcog.2021.107975.
28. He, S., Wiering, M. and Schomaker, L. (2015) ‘Junction detection in handwritten documents and its application to writer identification’, Pattern Recognition, Vol. 48, No. 12, pp.4036–4048, DOI: 10.1016/j.patcog.2015.05.022.
29. He, K., Zhang, X., Ren, S. and Sun, J. (2016) ‘Deep residual learning for image recognition’, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Las Vegas, NV, USA, pp.770–778, DOI: 10.1109/CVPR.2016.90.
30. Huang, C., Davis, L.S. and Townshend, J.R.G. (2002) ‘An assessment of support vector machines for land cover classification’, International Journal of Remote Sensing, Vol. 23, No. 4, pp.725–749, DOI: 10.1080/01431160110040323.
31. Huang, G., Liu, Z., Maaten, L.V.D. and Weinberger, K.Q. (2017) ‘Densely connected convolutional networks’, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Los Alamitos, CA, USA, pp.2261–2269, DOI: 10.1109/CVPR.2017.243.
32. Javidi, M. and Jampour, M. (2020) ‘A deep learning framework for text-independent writer identification’, Engineering Applications of Artificial Intelligence, Vol. 95, p.103912, DOI: 10.1016/j.engappai.2020.103912.
33. Lai, S., Zhu, Y. and Jin, L. (2020) ‘Encoding pathlet and SIFT features with bagged VLAD for historical writer identification’, IEEE Transactions on Information Forensics and Security, Vol. 15, pp.3553–3566, DOI: 10.1109/TIFS.2020.2991880.
34. Liashchynskyi, P. and Liashchynskyi, P. (2019) ‘Grid search, random search, genetic algorithm: a big comparison for NAS’, CoRR, DOI: abs/1912.06059.
35. Mahdianpari, M., Salehi, B., Rezaee, M., Mohammadimanesh, F. and Zhang, Y. (2018) ‘Very deep convolutional neural networks for complex land cover mapping using multispectral remote sensing imagery’, Remote Sensing, Vol. 10, No. 7, DOI: 10.3390/rs10071119.
36. Nasir, S. and Siddiqi, I. (2020) ‘Learning features for writer identification from handwriting on Papyri’, in 4th Mediterranean Conference on Pattern Recognition and Artificial Intelligence (MedPRAI), Proceedings, Communications in Computer and Information Science, Springer, Hammamet, Tunisia, Vol. 1322, 20–22 December, pp.229–241.
37. Ngo, T.T., Nguyen, H.T. and Nakagawa, M. (2021) ‘A-VLAD: an end-to-end attention-based neural network for writer identification in historical documents’, in 16th International Conference on Document Analysis and Recognition (ICDAR), Proceedings, Part II, Springer-Verlag, Berlin, Heidelberg, Lausanne, Switzerland, 5–10 September, pp.396–409, DOI: 10.1007/978-3-030-86331-9 26.
38. Nicolaou, A., Bagdanov, A.D., Liwicki, M. and Karatzas, D. (2015) ‘Sparse radial sampling LBP for writer identification’, in 13th International Conference on Document Analysis and Recognition (ICDAR), IEEE, Tunis, Tunisia, pp.716–720, DOI: 10.1109/ICDAR.2015.7333855.
39. Omayio, E., Sreedevi, I. and Panda, J. (2022) ‘Funnelling ensemble method for writer identification (Fem-Wi)’, SSRN Electronic Journal, DOI: 10.2139/ssrn.4022914.
40. Pal, M. (2005) ‘Random forest classifier for remote sensing classification’, International Journal of Remote Sensing, Vol. 26, No. 1, pp.217–222, DOI: 10.1080/01431160412331269698.
41. Phung, V.H. and Rhee, E.J. (2019) ‘A high-accuracy model average ensemble of convolutional neural networks for classification of cloud image patches on small datasets’, Applied Sciences, Vol. 9, No. 21, DOI: 10.3390/app9214500.
42. Rashad, M.Z., El-Bakry, H.M., Isma’il, I.R. and Mastorakis, N. (2010) ‘An overview of text-to-speech synthesis techniques’, in 4th International Conference on Communications and Information Technology (CIT) 2010, World Scientific and Engineering Academy and Society (WSEAS), Stevens Point, Wisconsin, USA, pp.84–89, DOI: 10.5555/1864098.1864113.
43. Rehman, A., Naz, S. and Razzak, M.I. (2018) ‘Writer identification using machine learning approaches: a comprehensive review’, Multimedia Tools and Applications, Vol. 78, pp.10889–10931, DOI: 10.1007/s11042-018-6577-1.
44. Rehman, A., Naz, S., Razzak, M.I. and Hameed, I.A. (2019) ‘Automatic visual features for writer identification: a deep learning approach’, IEEE Access, Vol. 7, pp.17149–17157, DOI: 10.1109/ACCESS.2018.2890810.
45. Ruder, S. (2016) ‘An overview of gradient descent optimization algorithms’, CoRR, DOI: abs/1609.04747.
46. Semma, A., Hannad, Y., Siddiqi, I., Djeddi, C. and El Youssfi El Kettani, M. (2021) ‘Writer identification using deep learning with FAST keypoints and Harris corner detector’, Expert Systems with Applications, Vol. 184, p.115473, DOI: 10.1016/j.eswa.2021.115473.
47. Shams, M., Elsonbaty, A.A. and El Sawy, W.Z. (2020) ‘Arabic handwritten character recognition based on convolution neural networks and support vector machine’, International Journal of Advanced Computer Science and Applications, Vol. 11, No. 8, pp.144–149, DOI: 10.14569/IJACSA.2020.0110819.
48. Simonyan, K. and Zisserman, A. (2015) ‘Very deep convolutional networks for large-scale image recognition’, in Bengio, Y. and LeCun, Y. (Eds.) 3rd International Conference on Learning Representations (ICLR), Conference Track Proceedings, San Diego, CA, USA, 7–9 May, http://arxiv.org/abs/1409.1556.
49. Sulaiman, A., Omar, K., Nasrudin, M.F. and Arram, A. (2019) ‘Length independent writer identification based on the fusion of deep and hand-crafted descriptors’, IEEE Access, Vol. 7, pp.91772–91784, DOI: 10.1109/ACCESS.2019.2927286.
50. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D. et al. (2015) ‘Going deeper with convolutions’, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Boston, MA, USA, pp.1–9, DOI: 10.1109/CVPR.2015.7298594.
51. Szegedy, C., Ioffe, S., Vanhoucke, V. and Alemi, A.A. (2017) ‘Inception-v4, Inception-ResNet and the impact of residual connections on learning’, in Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI), AAAI Press, San Francisco, California, USA, 4–9 February, pp.4278–4284.
52. Yang, J., Shokouhifar, M., Yee, P.L., Khan, A.A., Awais, M. and Mousavi, Z. (2024) ‘DT2F-TLNet: a novel text-independent writer identification and verification model using a combination of deep type-2 fuzzy architecture and transfer learning networks based on handwriting data’, Expert Systems with Applications, Vol. 242, p.122704, DOI: 10.1016/j.eswa.2023.122704.

Issue

Copyright Inderscience