Writing Order Recovery in Complex and Long Static Handwriting.
DOI:
https://doi.org/10.9781/ijimai.2021.04.003Keywords:
Clustering, Complex and Long Handwriting, Writing Order RecoveryAbstract
The order in which the trajectory is executed is a powerful source of information for recognizers. However, there is still no general approach for recovering the trajectory of complex and long handwriting from static images. Complex specimens can result in multiple pen-downs and in a high number of trajectory crossings yielding agglomerations of pixels (also known as clusters). While the scientific literature describes a wide range of approaches for recovering the writing order in handwriting, these approaches nevertheless lack a common evaluation metric. In this paper, we introduce a new system to estimate the order recovery of thinned static trajectories, which allows to effectively resolve the clusters and select the order of the executed pendowns. We evaluate how knowing the starting points of the pen-downs affects the quality of the recovered writing. Once the stability and sensitivity of the system is analyzed, we describe a series of experiments with three publicly available databases, showing competitive results in all cases. We expect the proposed system, whose code is made publicly available to the research community, to reduce potential confusion when the order of complex trajectories are recovered, and this will in turn make the trajectories recovered to be viable for further applications, such as velocity estimation.
Downloads
References
C. D. Stefano, F. Fontanella, A. Marcelli, R. Plamondon, “Graphonomics for the e-citizens: e-health, e-society and e-education,” 2019. doi: 10.1016/j.patrec.2018.11.020, Graphonomics for e-citizens: e-health, e-society, e-education.
R. Plamondon, S. N. Srihari, “Online and off-line handwriting recognition: a comprehensive survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 1, pp. 63–84, 2000, doi: 10.1109/34.824821.
R. Plamondon, G. Pirlo, E. Anquetil, C. Rémi, H. L. Teulings, M. Nakagawa, “Personal digital bodyguards for e-security, e-learning and e-health: A prospective survey,” Pattern Recognition, vol. 81, pp. 633–659, 2018.
C. Tomoiaga, P. Feng, M. Salzmann, P. Jayet, “Field typing for improved recognition on heterogeneous handwritten forms,” in 2019 International Conference on Document Analysis and Recognition (ICDAR), 2019, pp. 487–493, IEEE.
S. Dash, S. K. Shakyawar, M. Sharma, S. Kaushik, “Big data in healthcare: management, analysis and future prospects,” Journal of Big Data, vol. 6, no. 1, p. 54, 2019, doi: 10.1186/s40537-019-0217-0.
V. Rowtula, V. Bhargavan, M. Kumar, C. Jawahar, “Scaling handwritten student assessments with a document image workflow system,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2018, pp. 2307 – 2314.
R. Rajesh, R. Kanimozhi, “Digitized exam paper evaluation,” in 2019 IEEE International Conference on System, Computation, Automation and Networking (ICSCAN), 2019, pp. 1–5, IEEE.
I. H. Hsiao, “Mobile grading paper-based programming exams: automatic semantic partial credit assignment approach,” in European conference on technology enhanced learning, 2016, pp. 110–123, Springer.
A. Parziale, A. Della Cioppa, R. Senatore, A. Marcelli, “A decision tree for automatic diagnosis of parkinson's disease from offline drawing samples: experiments and findings,” in International Conference on Image Analysis and Processing, 2019, pp. 196–206, Springer.
M. Diaz, M. A. Ferrer, D. Impedovo, G. Pirlo, G. Vessio, “Dynamically enhanced static handwriting representation for parkinson's disease detection,” Pattern Recognition Letters, vol. 128, pp. 204–210, 2019, doi: 10.1016/j.patrec.2019.08.018.
S. Colutto, P. Kahle, H. Guenter, G. Muehlberger, “Transkribus. A platform for automated text recognition and searching of historical documents,” in 2019 15th International Conference on eScience (eScience), 2019, pp. 463–466, IEEE.
V. Nguyen, M. Blumenstein, “Techniques for static hand-writing trajectory recovery: a survey,” in Proceedings of the 9th IAPR International Workshop on Document Analysis Systems, 2010, pp. 463–470, ACM.
Z. Noubigh, M. Kherallah, “A survey on handwriting recognition based on the trajectory recovery technique,” in Arabic Script Analysis and Recognition (ASAR), 2017 1st International Workshop on, 2017, pp. 69–73, IEEE.
A. Hassaine, S. Al Maadeed, A. Bouridane, “ICDAR 2013 competition on handwriting stroke recovery from offline data,” in Proceedings of the International Conference on Document Analysis and Recognition, ICDAR, 2013.
C. De Stefano, A. Marcelli, A. Parziale, R. Senatore, “Reading cursive handwriting,” in 2010 12th International Conference on Frontiers in Handwriting Recognition, 2010, pp. 95–100, IEEE.
P. M. Lallican, C. Viard-Gaudin, S. Knerr, “From off-line to on-line handwriting recognition,” in Proceedings of the Seventh International Workshop on Frontiers in Handwriting Recognition, 2000, pp. 303–312. ISBN: 90-76942-01-3.
C. Viard-Gaudin, P. M. Lallican, S. Knerr, “Recognition-directed recovering of temporal information from hand-writing images,” Pattern Recognition Letters, vol. 26, no. 16, pp. 2537–2548, 2005, doi: 10.1016/j.patrec.2005.04.019.
B. Rabhi, A. Elbaati, H. Boubaker, A. M. Alimi, “Temporal order and pen velocity recovery for character handwriting based on sequence to sequence gated recurrent unit model,” TechRxiv, 2020.
L. Rousseau, É. Anquetil, J. Camillerapp, “Recovery of a drawing order from off-line isolated letters dedicated to on-line recognition,” in Proceedings of the International Conference on Document Analysis and Recognition, ICDAR, 2005.
A. K. Bhunia, A. Bhowmick, A. K. Bhunia, A. Konwer, P. Banerjee, P. P. Roy, U. Pal, “Handwriting trajectory recovery using end-to-end deep encoder-decoder network,” in 2018 24th International Conference on Pattern Recognition (ICPR), 2018, pp. 3639–3644, IEEE.
A. Sharma, “Recovery of drawing order in handwritten digit images,” in Image Information Processing (ICIIP), 2013 IEEE Second International Conference on, 2013, pp. 437–441, IEEE.
A. Santoro, A. Parziale, A. Marcelli, “A human in the loop approach to historical handwritten documents transcription,” in 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR), 2016, pp. 222–227, IEEE.
B. S. Saroui, V. Sorge, “Trajectory recovery and stroke reconstruction of handwritten mathematical symbols,” in Document Analysis and Recognition (ICDAR), 2015 13th International Conference on, 2015, pp. 1051–1055, IEEE.
M. Diaz, M. A. Ferrer, A. Parziale, A. Marcelli, “Recovering western on-line signatures from image-based specimens,” in 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), 2017, pp. 1204–1209, IEEE.
M. Diaz, M. A. Ferrer, D. Impedovo, M. I. Malik, G. Pirlo, R. Plamondon, “A perspective analysis of handwritten signature technology,” ACM Computing Surveys (CSUR), vol. 51, no. 6, pp. 1–39, 2019, doi: 10.1145/3274658.
A. Parziale, A. Santoro, A. Marcelli, A. P. Rizzo, C. Molinari, A. G. Cappuzzo, F. Fontana, “An interactive tool for forensic handwriting examination,” in 2014 14th International Conference on Frontiers in Handwriting Recognition, 2014, pp. 440–445, IEEE.
A. Marcelli, A. Parziale, C. De Stefano, “Quantitative evaluation of features for forensic handwriting examination,” in 2015 13th International Conference on Document Analysis and Recognition (ICDAR), 2015, pp. 1266–1271, IEEE.
A. Marcelli, A. Parziale, A. Santoro, “Modeling handwriting style: a preliminary investigation,” in 2012 International Conference on Frontiers in Handwriting Recognition, 2012, pp. 411–416, IEEE.
A. Marcelli, A. Parziale, A. Santoro, “Modelling visual appearance of handwriting,” in International Conference on Image Analysis and Processing, 2013, pp. 673–682, Springer.
C. Carmona-Duarte, M. A. Ferrer, A. Parziale, A. Marcelli, “Temporal evolution in synthetic handwriting,” Pattern Recognition, vol. 68, pp. 233– 244, 2017, doi: 10.1016/j.patcog.2017.03.019.
A. Marcelli, A. Parziale, R. Senatore, “Some observations on handwriting from a motor learning perspective.,” in 2nd International Workshop on Automated Forensic Handwriting Analysis, 2013, pp. 6–10.
M. Faundez-Zanuy, J. Fierrez, M. A. Ferrer, M. Diaz, R. Tolosana, R. Plamondon, “Handwriting biometrics: Applications and future trends in e-security and e-health,” Cognitive Computation, vol. 12, no. 5, pp. 940–953, 2020, doi: 10.1007/s12559-020-09755-z.
S. Al-Maadeed, W. Ayouby, A. Hassaine, A. Almejali, A. Al-yazeedi, R. Al-Atiya, “Arabic signature verification dataset,” in Proceedings of the International Arab Conference on Information Technology, 2012. ISSN: 1812-0857.
R. Plamondon, C. M. Privitera, “The segmentation of cursive handwriting: an approach based on off-line recovery of the motor-temporal information,” IEEE Transactions on Image Processing, vol. 8, no. 1, pp. 80–91, 1999, doi: 10.1109/83.736691.
B. Kovari, “Time-efficient stroke extraction method for handwritten signatures,” in Proceedings of the 7th International Conference on Applied Computer Science - Volume 7, ACS'07, Stevens Point, Wisconsin, USA, 2007, pp. 157–161, World Scientific and Engineering Academy and Society (WSEAS). ISBN: 9789606766183.
G. Boccignone, A. Chianese, L. P. Cordella, A. Marcelli, “Recovering dynamic information from static handwriting,” Pattern recognition, vol. 26, no. 3, pp. 409–418, 1993, doi: 10.1016/0031-3203(93)90168-V.
S. Lee, J. C. Pan, “Offline tracing and representation of signatures,” IEEE Transactions on Systems, Man, and Cybernetics, vol. 22, no. 4, pp. 755–771, 1992, doi: 10.1109/21.156588.
S. Lee, J. C. Pan, “Handwritten numeral recognition based on hierarchically self-organizing learning networks,” in IEEE International Joint Conference on Neural Networks, 1991, pp. 1313–1322, IEEE.
D. S. Doermann, A. Rosenfeld, “Recovery of temporal information from static images of handwriting,” International Journal of Computer Vision, vol. 15, no. 1-2, pp. 143–164, 1995, doi: 10.1007/BF01450853.
J. Hennebert, R. Loeffel, A. Humm, R. Ingold, “A new forgery scenario based on regaining dynamics of signature,” in International Conference on Biometrics, 2007, pp. 366–375, Springer.
A. Elbaati, M. Kherallah, A. Ennaji, A. M. Alimi, “Temporal order recovery of the scanned handwriting,” in Proceedings of the International Conference on Document Analysis and Recognition, ICDAR, 2009.
S. Jager, “Recovering writing traces in off-line handwriting recognition: using a global optimization technique,” in Proceedings of 13th International Conference on Pattern Recognition, vol. 3, 1996, pp. 150–154 vol.3.
H. Bunke, R. Ammann, G. Kaufmann, T. M. Ha, M. Schenkel, R. Seiler, F. Eggimann, “Recovery of temporal information of cursively handwritten words for on-line recognition,” in Proceedings of the Fourth International Conference on Document Analysis and Recognition, vol. 2, 1997, pp. 931–935, IEEE.
Y. Kato, M. Yasuhara, “Recovery of drawing order from single-stroke handwriting images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 9, pp. 938–949, 2000, doi: 10.1109/34.877517.
Y. Qiao, M. Nishiara, M. Yasuhara, “A framework toward restoration of writing order from single-stroked handwriting image,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 11, pp. 1724–1737, 2006, doi: 10.1109/TPAMI.2006.216.
L. P. Cordella, C. De Stefano, A. Marcelli, A. Santoro, “Writing order recovery from off-line handwriting by graph traversal,” in 20th International Conference on Pattern Recognition (ICPR), 2010, pp. 1896–1899, IEEE.
R. Senatore, A. Santoro, A. Marcelli, “From motor to trajectory plan: A feedback loop between unfolding and segmentation to improve writing order recovery,” in 15th International Graphonomics Society Conference, Cancun, Mexico, 2011, pp. 86–69. ISBN: 9780732640033.
M. Dinh, H. J. Yang, G. S. Lee, S. H. Kim, L. N. Do, “Recovery of drawing order from multi-stroke english handwritten images based on graph models and ambiguous zone analysis,” Expert Systems with Applications, vol. 64, pp. 352–364, 2016, doi: 10.1016/j.eswa.2016.08.017.
C. Viard-Gaudin, P. M. Lallican, S. Knerr, P. Binter, “The IRESTE on/off (IRONOFF) dual handwriting database,” in Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR'99 (Cat. No. PR00318), 1999, pp. 455–458, IEEE.
G. Crispo, M. Diaz, A. Marcelli, M. A. Ferrer, “Tracking the ballistic trajectory in complex and long handwritten signatures,” in 16th International Conference on Frontiers in Handwriting Recognition, 2018, pp. 351–356.
E. M. Nel, J. A. Du Preez, B. M. Herbst, “Estimating the pen trajectories of static signatures using hidden markov models,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 11, pp. 1733–1746, 2005, doi: 10.1109/TPAMI.2005.221.
K. K. Lau, P. C. Yuen, Y. Y. Tang, “Universal writing model for recovery of writing sequence of static handwriting images,” International Journal of Pattern Recognition and Artificial Intelligence, vol. 19, no. 05, pp. 603–630, 2005, doi: 10.1142/S0218001405004277.
B. Zhao, M. Yang, J. Tao, “Pen tip motion prediction for handwriting drawing order recovery using deep neural network,” in 2018 24th International Conference on Pattern Recognition (ICPR), Aug 2018, pp. 704–709.
B. Zhao, M. Yang, J. Tao, “Drawing order recovery for handwriting chinese characters,” in ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 2019, pp. 3227–3231.
B. Rabhi, A. Elbaati, Y. Hamdi, A. M. Alimi, “Handwriting recognition based on temporal order restored by the end-to-end system,” in International Conference on Document Analysis and Recognition (ICDAR), 2019, pp. 1231–1236.
T. Sumi, B. K. Iwana, H. Hayashi, S. Uchida, “Modality conversion of handwritten patterns by cross variational autoencoders,” in 2019 International Conference on Document Analysis and Recognition (ICDAR), 2019, pp. 407–412.
R. Zhang, J. Chen, M. Yang, “Drawing order recovery based on deep learning,” in 2019 Eleventh International Conference on Advanced Computational Intelligence (ICACI), 2019, pp. 129–133, IEEE.
H. T. Nguyen, T. Nakamura, C. T. Nguyen, M. Nakagawa, “Online trajectory recovery from offline handwritten japanese kanji characters of multiple strokes,” in 25th International Conference on Pattern Recognition (ICPR), 2020, IEEE.
C. De Stefano, M. Garruto, A. Marcelli, “A saliency-based multiscale method for on-line cursive handwriting shape description,” International Journal of Pattern Recognition and Artificial Intelligence, vol. 18, no. 6, pp. 1139–1156, 2004. ISSN:0218-0014.
E. W. Dijkstra, “A note on two problems in connexion with graphs,” Numerische mathematik, vol. 1, no. 1, pp. 269–271, 1959, doi: 10.1007/ BF01386390.
F. H. Allport, Theories of perception and the concept of structure: A review and critical analysis with an introduction to a dynamic-structural theory of behavior. John Wiley & Sons Inc, 1955.
R. Plamondon, “A kinematic theory of rapid human movements: Part III. kinetic outcomes,” Biological Cybernetics, vol. 78, no. 2, pp. 133–145, 1998, doi: 10.1007/s004220050420.
T. Steinherz, D. Doermann, E. Rivlin, N. Intrator, “Offline loop investigation for handwriting analysis,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 2, pp. 193–209, 2008, doi: 10.1109/TPAMI.2008.68.
V. L. Blankers, C. E. van den Heuvel, K. Y. Franke, L. G. Vuurpijl, “ICDAR 2009 signature verification competition,” in 2009 10th International Conference on Document Analysis and Recognition, 2009, pp. 1403–1407, IEEE.
J. E. Bresenham, “Algorithm for computer control of a digital plotter,” IBM Systems Journal, vol. 4, no. 1, pp. 25–30, 1965, doi: 10.1147/sj.41.0025.
M. A. Ferrer, M. Diaz, C. Carmona-Duarte, R. Plamondon, “idelog: Iterative dual spatial and kinematic extraction of sigmalognormal parameters,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 1, pp. 114–125, 2020, doi: 10.1109/TPAMI.2018.2879312.
M. Diaz, P. Henriquez, M. A. Ferrer, G. Pirlo, J. B. Alonso, C. Carmona Duarte, D. Impedovo, “Stability-based system for bearing fault early detection,” Expert Systems with Applications, vol. 79, pp. 65–75, 2017, doi: 10.1016/j.eswa.2017.02.030.
A. Kholmatov, B. Yanikoglu, “SUSIG: an on-line signature database, associated protocols and benchmark results,” Pattern Analysis and Applications, vol. 12, no. 3, pp. 227–236, 2009, doi: 10.1007/s10044-008- 0118-x.
D. Y. Yeung, H. Chang, Y. Xiong, S. George, R. Kashi, T. Matsumoto, G. Rigoll, “SVC2004: First international signature verification competition,” in International conference on biometric authentication, 2004, pp. 16–22, Springer.
Downloads
Published
-
Abstract188
-
PDF30
Submit Article
Information
