Endre søk
Link to record
Permanent link

Direct link
Nikolaidou, KonstantinaORCID iD iconorcid.org/0000-0002-9332-3188
Publikasjoner (10 av 14) Visa alla publikasjoner
Nikolaidou, K., Retsinas, G., Sfikas, G. & Liwicki, M. (2025). DiffusionPen: Towards Controlling the Style of Handwritten Text Generation. In: Aleš Leonardis; Elisa Ricci; Stefan Roth; Olga Russakovsky; Torsten Sattler; Gül Varol (Ed.), Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4, 2024, Proceedings, Part LXXXV. Paper presented at 18th European Conference on Computer Vision (ECCV 2024), Milano, Italy, September 29 - October 4, 2024 (pp. 417-434). Springer Science and Business Media Deutschland GmbH, LXXXV
Åpne denne publikasjonen i ny fane eller vindu >>DiffusionPen: Towards Controlling the Style of Handwritten Text Generation
2025 (engelsk)Inngår i: Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4, 2024, Proceedings, Part LXXXV / [ed] Aleš Leonardis; Elisa Ricci; Stefan Roth; Olga Russakovsky; Torsten Sattler; Gül Varol, Springer Science and Business Media Deutschland GmbH , 2025, Vol. LXXXV, s. 417-434Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

Handwritten Text Generation (HTG) conditioned on text and style is a challenging task due to the variability of inter-user characteristics and the unlimited combinations of characters that form new words unseen during training. Diffusion Models have recently shown promising results in HTG but still remain under-explored. We present DiffusionPen (DiffPen), a 5-shot style handwritten text generation approach based on Latent Diffusion Models. By utilizing a hybrid style extractor that combines metric learning and classification, our approach manages to capture both textual and stylistic characteristics of seen and unseen words and styles, generating realistic handwritten samples. Moreover, we explore several variation strategies of the data with multi-style mixtures and noisy embeddings, enhancing the robustness and diversity of the generated data. Extensive experiments using IAM offline handwriting database show that our method outperforms existing methods qualitatively and quantitatively, and its additional generated data can improve the performance of Handwriting Text Recognition (HTR) systems.

sted, utgiver, år, opplag, sider
Springer Science and Business Media Deutschland GmbH, 2025
Serie
Lecture Notes in Computer Science (LNCS), ISSN 0302-9743, E-ISSN 1611-3349 ; 15143
Emneord
Handwriting Generation, Latent Diffusion Models, Few-shot Style Representation
HSV kategori
Forskningsprogram
Maskininlärning
Identifikatorer
urn:nbn:se:ltu:diva-111074 (URN)10.1007/978-3-031-73013-9_24 (DOI)001416939500024 ()2-s2.0-85211230972 (Scopus ID)
Konferanse
18th European Conference on Computer Vision (ECCV 2024), Milano, Italy, September 29 - October 4, 2024
Merknad

ISBN for host publication: 978-3-031-73012-2, 978-3-031-73013-9

Tilgjengelig fra: 2024-12-17 Laget: 2024-12-17 Sist oppdatert: 2025-10-21bibliografisk kontrollert
Nikolaidou, K., Retsinas, G., Sfikas, G., Cascianelli, S., Cucchiara, R. & Liwicki, M. (2025). Dual Orthogonal Guidance for Robust Diffusion-Based Handwritten Text Generation. In: Proceedings - 2025 IEEE/CVF International Conference on Computer Vision Workshops, ICCV-W 2025: . Paper presented at 2025 International Conference on Computer Vision Workshops (ICCVW), Honolulu, Hawaii, USA, October 19-23, 2025 (pp. 7020-7029). Institute of Electrical and Electronics Engineers Inc.
Åpne denne publikasjonen i ny fane eller vindu >>Dual Orthogonal Guidance for Robust Diffusion-Based Handwritten Text Generation
Vise andre…
2025 (engelsk)Inngår i: Proceedings - 2025 IEEE/CVF International Conference on Computer Vision Workshops, ICCV-W 2025, Institute of Electrical and Electronics Engineers Inc. , 2025, s. 7020-7029Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

Diffusion-based Handwritten Text Generation (HTG) approaches achieve impressive results on frequent, invocabulary words observed at training time and on regular styles. However, they are prone to memorizing training samples and often struggle with style variability and generation clarity. In particular, standard diffusion models tend to produce artifacts or distortions that negatively affect the readability of the generated text, especially when the style is hard to produce. To tackle these issues, we propose a novel sampling guidance strategy, Dual Orthogonal Guidance (DOG), that leverages an orthogonal projection of a negatively perturbed prompt onto the original positive prompt. This approach helps steer the generation away from artifacts while maintaining the intended content, and encourages more diverse, yet plausible, outputs. Unlike standard Classifier-Free Guidance (CFG), which relies on unconditional predictions and produces noise at high guidance scales, DOG introduces a more stable, disentangled direction in the latent space. To control the strength of the guidance across the denoising process, we apply a triangular schedule: weak at the start and end of denoising, when the process is most sensitive, and strongest in the middle steps. Experimental results on the state-of-the-art DiffusionPen and One-DM demonstrate that DOG improves both content clarity and style variability, even for out-of-vocabulary words and challenging writing styles.

sted, utgiver, år, opplag, sider
Institute of Electrical and Electronics Engineers Inc., 2025
HSV kategori
Forskningsprogram
Maskininlärning
Identifikatorer
urn:nbn:se:ltu:diva-117497 (URN)10.1109/ICCVW69036.2025.00725 (DOI)001740020100725 ()2-s2.0-105035150466 (Scopus ID)
Konferanse
2025 International Conference on Computer Vision Workshops (ICCVW), Honolulu, Hawaii, USA, October 19-23, 2025
Forskningsfinansiär
Knut and Alice Wallenberg Foundation
Merknad

ISBN for host publication: 979-8-3315-8988-2

Tilgjengelig fra: 2026-06-05 Laget: 2026-06-05 Sist oppdatert: 2026-06-05bibliografisk kontrollert
Nikolaidou, K. (2025). Handwritten Text Generation with Diffusion Models: Beyond Visual Quality. (Doctoral dissertation). Luleå: Luleå University of Technology
Åpne denne publikasjonen i ny fane eller vindu >>Handwritten Text Generation with Diffusion Models: Beyond Visual Quality
2025 (engelsk)Doktoravhandling, monografi (Annet vitenskapelig)
Abstract [en]

Handwritten Text Generation (HTG) has emerged as a promising remedy for data scarcity that limits the training of Deep Learning (DL) models for Document Image Analysis and Recognition (DIAR) tasks. Current HTG pipelines are mostly based on adversarial training, which can face several issues, such as mode collapse, limiting variability, and in turn, reducing the practical usefulness of the generated data. Moreover, existing evaluation protocols prioritize visual realism rather than usefulness, obscuring the link between generative quality and downstream task performance.

These limitations raise several questions related to which HTG methods truly benefit DIAR tasks, which real datasets should serve as a foundation for synthetic data generation, how Diffusion Models can be adapted for controllable and robust HTG, and whether current evaluation protocols capture downstream utility. Given the success of text-to-image Diffusion Models in generating realistic images from text prompts, their adaptation for handwriting generation offers the potential to produce high-quality, style-aware handwritten data that can directly enhance Handwritten Text Recognition (HTR). The overarching goal of this thesis is to bridge the gap between generation and task-aligned evaluation by showing how controllable diffusion-based HTG, guided by systematic data analysis and evaluated through practical utility, can address these challenges.

The contributions of this thesis are organized along three complementary directions. First, to establish a solid basis for HTG, a comprehensive overview of modern, historical, and synthetic document image datasets and HTG methods is presented. This results in C1, a systematic overview of dataset resources, and C2, a detailed survey of generative paradigms and evaluation practices in HTG, identifying key data and methodological gaps that motivate the development of diffusion-based models. Second, to overcome the instability and limited variability of adversarial methods, three diffusion-based approaches are proposed. C3 (WordStylist), a latent diffusion model enabling verbatim text and style conditioning. C4 (DiffusionPen), a few-shot extension of WordStylist capable of generalizing to unseen writers through hybrid classification and metric-learning style embeddings. Moreover, C5 (Dual Orthogonal Guidance), a sampling-time mechanism that enhances stability while preserving stylistic diversity. These proposed HTG methods demonstrate that diffusion-based models can generate realistic, diverse, and style-consistent handwriting under controllable conditions. Third, recognizing that generative quality should translate into practical utility, C6 introduces a task-aligned evaluation framework that links generation metrics to recognition outcomes. This framework measures content preservation, style preservation, robustness to Out-of-Vocabulary (OOV) content, and variability, providing a practical assessment of whether synthetic handwriting improves DIAR performance. By integrating generation and evaluation, this contribution redefines how HTG success is measured.

In summary, this thesis demonstrates that controllable diffusion-based HTG, grounded in systematic dataset analysis, enabled by robust generative modeling, and evaluated with task-aligned metrics, provides efficient synthetic handwriting pipelines that directly enhance HTR performance. On a broader level, this work bridges the gap between generative modeling and document analysis, setting the stage for future research in task-aware synthesis of document images.

sted, utgiver, år, opplag, sider
Luleå: Luleå University of Technology, 2025
Serie
Doctoral thesis / Luleå University of Technology, ISSN 1402-1544
Emneord
handwritten text generation, diffusion models
HSV kategori
Forskningsprogram
Maskininlärning
Identifikatorer
urn:nbn:se:ltu:diva-115100 (URN)978-91-8048-923-2 (ISBN)978-91-8048-924-9 (ISBN)
Disputas
2025-12-09, A117, Luleå University of Technology, Luleå, 09:00 (engelsk)
Opponent
Veileder
Forskningsfinansiär
Swedish Research Council, 363131
Tilgjengelig fra: 2025-10-14 Laget: 2025-10-13 Sist oppdatert: 2025-11-18bibliografisk kontrollert
Pippi, V., Nikolaidou, K., Cascianelli, S., Retsinas, G., Sfikas, G., Cucchiara, R. & Liwicki, M. (2025). Quo Vadis Handwritten Text Generation for Handwritten Text Recognition?. In: Proceedings - 2025 IEEE/CVF International Conference on Computer Vision Workshops, ICCV-W 2025: . Paper presented at 2025 International Conference on Computer Vision Workshops (ICCVW), Honolulu, Hawaii, USA, October 19-23, 2025 (pp. 7523-7533). Institute of Electrical and Electronics Engineers Inc.
Åpne denne publikasjonen i ny fane eller vindu >>Quo Vadis Handwritten Text Generation for Handwritten Text Recognition?
Vise andre…
2025 (engelsk)Inngår i: Proceedings - 2025 IEEE/CVF International Conference on Computer Vision Workshops, ICCV-W 2025, Institute of Electrical and Electronics Engineers Inc. , 2025, s. 7523-7533Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

The digitization of historical manuscripts presents significant challenges for Handwritten Text Recognition (HTR) systems, particularly when dealing with small, author-specific collections that diverge from the training data distributions. Handwritten Text Generation (HTG) techniques, which generate synthetic data tailored to specific handwriting styles, offer a promising solution to address these challenges. However, the effectiveness of various HTG models in enhancing HTR performance, especially in low-resource transcription settings, has not been thoroughly evaluated. In this work, we systematically compare three state-of-the-art styled HTG models (representing the generative adversarial, diffusion, and autoregressive paradigms for HTG) to assess their impact on HTR fine-tuning. We analyze how visual and linguistic characteristics of synthetic data influence fine-tuning outcomes and provide quantitative guide-lines for selecting the most effective HTG model. The results of our analysis provide insights into the current capabilities of HTG methods and highlight key areas for further improvement in their application to low-resource HTR.

sted, utgiver, år, opplag, sider
Institute of Electrical and Electronics Engineers Inc., 2025
HSV kategori
Forskningsprogram
Maskininlärning
Identifikatorer
urn:nbn:se:ltu:diva-117498 (URN)10.1109/ICCVW69036.2025.00775 (DOI)001740020100773 ()2-s2.0-105035151586 (Scopus ID)
Konferanse
2025 International Conference on Computer Vision Workshops (ICCVW), Honolulu, Hawaii, USA, October 19-23, 2025
Merknad

ISBN for host publication: 979-8-3315-8988-2

Tilgjengelig fra: 2026-06-05 Laget: 2026-06-05 Sist oppdatert: 2026-06-05bibliografisk kontrollert
Nikolaidou, K., Retsinas, G., Sfikas, G. & Liwicki, M. (2025). Rethinking HTG Evaluation: Bridging Generation and Recognition. In: Alessio Del Bue, Cristian Canton, Jordi Pont-Tuset, Tatiana Tommasi (Ed.), Computer Vision – ECCV 2024 Workshops: . Paper presented at European Conference on Computer Vision, ECCV 2024 Workshops, September 29 – October 4, 2024, Milan, Italy (pp. 179-195). Springer Science and Business Media Deutschland GmbH, Part XXII
Åpne denne publikasjonen i ny fane eller vindu >>Rethinking HTG Evaluation: Bridging Generation and Recognition
2025 (engelsk)Inngår i: Computer Vision – ECCV 2024 Workshops / [ed] Alessio Del Bue, Cristian Canton, Jordi Pont-Tuset, Tatiana Tommasi, Springer Science and Business Media Deutschland GmbH , 2025, Vol. Part XXII, s. 179-195Konferansepaper, Publicerat paper (Fagfellevurdert)
sted, utgiver, år, opplag, sider
Springer Science and Business Media Deutschland GmbH, 2025
Serie
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349 ; 15644
HSV kategori
Forskningsprogram
Maskininlärning
Identifikatorer
urn:nbn:se:ltu:diva-113555 (URN)10.1007/978-3-031-92089-9_12 (DOI)001544995100012 ()2-s2.0-105006912334 (Scopus ID)
Konferanse
European Conference on Computer Vision, ECCV 2024 Workshops, September 29 – October 4, 2024, Milan, Italy
Merknad

ISBN for host publication: 978-3-031-92088-2, 978-3-031-92089-9;

Tilgjengelig fra: 2025-06-18 Laget: 2025-06-18 Sist oppdatert: 2025-11-28bibliografisk kontrollert
Retsinas, G., Nikolaidou, K. & Sfikas, G. (2024). Enhancing CRNN HTR Architectures with Transformer Blocks. In: Elisa H. Barney Smith; Marcus Liwicki; Liangrui Peng (Ed.), Document Analysis and Recognition, ICDAR 2024: 18th International Conference, Athens, Greece, August 30–September 4, 2024, Proceedings, Part IV. Paper presented at 18th International Conference on Document Analysis and Recognition (ICDAR 2024), Athens, Greece, August 30–September 4, 2024 (pp. 425-440). Springer Science and Business Media Deutschland GmbH, 4
Åpne denne publikasjonen i ny fane eller vindu >>Enhancing CRNN HTR Architectures with Transformer Blocks
2024 (engelsk)Inngår i: Document Analysis and Recognition, ICDAR 2024: 18th International Conference, Athens, Greece, August 30–September 4, 2024, Proceedings, Part IV / [ed] Elisa H. Barney Smith; Marcus Liwicki; Liangrui Peng, Springer Science and Business Media Deutschland GmbH , 2024, Vol. 4, s. 425-440Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

Handwritten Text Recognition (HTR) is a challenging problem that plays an essential role in digitizing and interpreting diverse handwritten documents. While traditional approaches primarily utilize CNN-RNN (CRNN) architectures, recent advancements based on Transformer architectures have demonstrated impressive results in HTR. However, these Transformer-based systems often involve high-parameter configurations and rely extensively on synthetic data. Moreover, they lack focus on efficiently integrating the ability of Transformer modules to grasp contextual relationships within the data. In this paper, we explore a lightweight integration of Transformer modules into existing CRNN frameworks to address the complexities of HTR, aiming to enhance the context of the sequential nature of the task. We present a hybrid CNN image encoder with intermediate MobileViT blocks that effectively combines the different components in a resource-efficient manner. Through extensive experiments and ablation studies, we refine the integration of these modules and demonstrate that our proposed model enhances HTR performance. Our results on the line-level IAM and RIMES datasets suggest that our proposed method achieves competitive performance with significantly fewer parameters and without integrating synthetic data compared to existing systems. 

sted, utgiver, år, opplag, sider
Springer Science and Business Media Deutschland GmbH, 2024
Serie
Lecture Notes in Computer Science, ISSN 1611-3349, E-ISSN 0302-9743
Emneord
Handwriting Text Recognition, Transformer Modules
HSV kategori
Forskningsprogram
Maskininlärning
Identifikatorer
urn:nbn:se:ltu:diva-110168 (URN)10.1007/978-3-031-70546-5_25 (DOI)001336396200025 ()2-s2.0-85204589724 (Scopus ID)
Konferanse
18th International Conference on Document Analysis and Recognition (ICDAR 2024), Athens, Greece, August 30–September 4, 2024
Merknad

ISBN for host publication: 978-3-031-70545-8, 978-3-031-70546-5

Tilgjengelig fra: 2024-09-30 Laget: 2024-09-30 Sist oppdatert: 2025-10-21bibliografisk kontrollert
Saleh, Y. S., Mokayed, H., Nikolaidou, K., Alkhaled, L. & Hum, Y. C. (2024). How GANs assist in Covid-19 pandemic era: a review. Multimedia tools and applications, 83(10), 29915-29944
Åpne denne publikasjonen i ny fane eller vindu >>How GANs assist in Covid-19 pandemic era: a review
Vise andre…
2024 (engelsk)Inngår i: Multimedia tools and applications, ISSN 1380-7501, E-ISSN 1573-7721, Vol. 83, nr 10, s. 29915-29944Artikkel, forskningsoversikt (Fagfellevurdert) Published
sted, utgiver, år, opplag, sider
Springer Nature, 2024
HSV kategori
Forskningsprogram
Maskininlärning; Centrumbildning - ProcessIT Innovations
Identifikatorer
urn:nbn:se:ltu:diva-101620 (URN)10.1007/s11042-023-16597-y (DOI)001182559400057 ()2-s2.0-85170832380 (Scopus ID)
Merknad

Validerad;2024;Nivå 2;2024-04-09 (hanlid)

Tilgjengelig fra: 2023-10-11 Laget: 2023-10-11 Sist oppdatert: 2025-10-21bibliografisk kontrollert
Carrière, G., Nikolaidou, K., Kordon, F., Mayr, M., Seuret, M. & Christlein, V. (2023). Beyond Human Forgeries: An Investigation into Detecting Diffusion-Generated Handwriting. In: Mickael Coustaty; Alicia Fornés (Ed.), Document Analysis and Recognition – ICDAR 2023 Workshops, Part I: . Paper presented at 17th International Conference on Document Analysis and Recognition (ICDAR 2023), San José, CA, United States, August 21-26,2023 (pp. 5-19). Springer
Åpne denne publikasjonen i ny fane eller vindu >>Beyond Human Forgeries: An Investigation into Detecting Diffusion-Generated Handwriting
Vise andre…
2023 (engelsk)Inngår i: Document Analysis and Recognition – ICDAR 2023 Workshops, Part I / [ed] Mickael Coustaty; Alicia Fornés, Springer, 2023, s. 5-19Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

Methods for detecting forged handwriting are usually based on the assumption that the forged handwriting is produced by humans. Authentic-looking handwriting, however, can also be produced synthetically. Diffusion-based generative models have recently gained popularity as they produce striking natural images and are also able to realistically mimic a person’s handwriting. It is, therefore, reasonable to assume that these models will be used to forge handwriting in the near future, adding a new layer to handwriting forgery detection. We show for the first time that the identification of synthetic handwritten data is possible by a small Convolutional Neural Network (ResNet18) reaching accuracies of 90%. We further investigate the existence of distinct discriminative features in synthetic handwriting data produced by latent diffusion models that could be exploited to build stronger detection methods. Our experiments indicate that the strongest discriminative features do not come from generation artifacts, letter shapes, or the generative model’s architecture, but instead originate from real-world artifacts in genuine handwriting that are not reproduced by generative methods.

sted, utgiver, år, opplag, sider
Springer, 2023
Serie
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349 ; 14193
Emneord
Biometrics, Diffusion Models, Forensics Analysis, Forgery Detection, Handwriting Generation, Synthetic Image Generation
HSV kategori
Forskningsprogram
Maskininlärning
Identifikatorer
urn:nbn:se:ltu:diva-103374 (URN)10.1007/978-3-031-41498-5_1 (DOI)001346410700001 ()2-s2.0-85173041756 (Scopus ID)
Konferanse
17th International Conference on Document Analysis and Recognition (ICDAR 2023), San José, CA, United States, August 21-26,2023
Merknad

ISBN for host publication: 978-3-031-41497-8 (print), 978-3-031-41498-5 (electronic)

Tilgjengelig fra: 2024-01-03 Laget: 2024-01-03 Sist oppdatert: 2025-10-21bibliografisk kontrollert
Nikolaidou, K. (2023). Enabling Deep Document Image Analysis with Generative Models. (Licentiate dissertation). Luleå: Luleå University of Technology
Åpne denne publikasjonen i ny fane eller vindu >>Enabling Deep Document Image Analysis with Generative Models
2023 (engelsk)Licentiatavhandling, med artikler (Annet vitenskapelig)
Abstract [en]

Historical documents are a valuable source of cultural knowledge and can provide information about previous events, societies, beliefs, and cultures. They can serve as an excellent source for research in various fields including history, literature, linguistics, and anthropology. Their preservation and analysis pose significant challenges due to the unique characteristics of handwritten scripts, the variability, and the document degradation. With the rise of the Deep Learning era, enormous amounts of annotated data are required to train large models that can efficiently perform tasks on unseen data. Nowadays, digital libraries provide high-quality digitized images for analysis and processing of historical documents. However, collecting and annotating the provided data is an expensive task and requires a lot of expertise from historians and the humanities. Hence, generating synthetic data to enhance the performance of Deep Learning frameworks is a common approach in Computer Vision and, specifically in this thesis, in Document Image Analysis and Recognition (DIAR).

This thesis focuses on leveraging generative models to facilitate DIAR tasks, focusing on historical and handwritten documents, by generating realistic synthetic images that resemble a real distribution and enhance the training of downstream DIAR tasks. The contributions of the thesis include a systematic literature review, a comparison evaluation, and a developed method for handwriting generation.

First, a systematic literature review of existing historical document image datasets, provides summarized information of 65 studies, focusing on different aspects, such as statistics, document type, language, visual, and annotation aspects. The study discusses limitations and promising resources for future research, which refer to the limited dataset size and absence of benchmarks, as well as the lack of standardization in terms of data format and evaluation scheme.

A subsequent contribution is the integration of generated data in a historical document font classification task. Semi-synthetic data are generated with the use of DocCreator, an open-source software, from which different document degradation augmentations are used. A conditional Generative Adversarial Network (GAN) is used to generate fully synthetic data conditioned on a specific sample. The data generated by the two methods areintegrated as additional samples in the training of several Convolutional Neural Networks classifiers and the effect in the performance is examined.

The final contribution of the thesis introduces a new method for generating styled handwritten text images based on Denoising Diffusion Probabilistic Models (DDPM), which is an unexplored method in DIAR. The method manages to capture stylistic and content characteristics of a standard multi-writer handwriting dataset and achieved an improved performance in enhancing writer identification and handwriting text recognition compared to Generative Adversarial Network (GAN)-based methods. The results demonstrate the potential of the generative method for enabling deep document image analysis and pave the way for further research.

As a future direction, this work will aim to progress from generating word images to generating sentence and full document images by conditioning on the content, style, and layout of historical documents. Another future action will be to further extend the proposed method to operate in a few-shot scheme for the writer style condition in order to generate unseen styles. Furthermore, the future work will aim to leverage important features from pre-training with synthetic and real data in order to generalize to historical documents that are a scarce source and adjusting the text encoding parts to different languages and scripts. Finally, the ultimate goal of the future work aims to generate a massive synthetic historical document image database to fill the existing benchmark gap.

sted, utgiver, år, opplag, sider
Luleå: Luleå University of Technology, 2023
Serie
Licentiate thesis / Luleå University of Technology, ISSN 1402-1757
HSV kategori
Forskningsprogram
Maskininlärning
Identifikatorer
urn:nbn:se:ltu:diva-96361 (URN)978-91-8048-303-2 (ISBN)978-91-8048-304-9 (ISBN)
Presentation
2023-06-07, A117, Luleå tekniska universitet, Luleå, 10:00 (engelsk)
Opponent
Veileder
Tilgjengelig fra: 2023-04-12 Laget: 2023-04-11 Sist oppdatert: 2025-10-21bibliografisk kontrollert
Nikolaidou, K., Retsinas, G., Christlein, V., Seuret, M., Sfikas, G., Barney Smith, E., . . . Liwicki, M. (2023). WordStylist: Styled Verbatim Handwritten Text Generation with Latent Diffusion Models. In: Gernot A. Fink, Rajiv Jain, Koichi Kise & Richard Zanibbi (Ed.), Document Analysis and Recognition - ICDAR 2023, Part II: . Paper presented at 17th International Conference on Document Analysis and Recognition (ICDR2023), San José, CA, USA, August 21-26. 2023 (pp. 384-401). Springer
Åpne denne publikasjonen i ny fane eller vindu >>WordStylist: Styled Verbatim Handwritten Text Generation with Latent Diffusion Models
Vise andre…
2023 (engelsk)Inngår i: Document Analysis and Recognition - ICDAR 2023, Part II / [ed] Gernot A. Fink, Rajiv Jain, Koichi Kise & Richard Zanibbi, Springer, 2023, s. 384-401Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

Text-to-Image synthesis is the task of generating an image according to a specific text description. Generative Adversarial Networks have been considered the standard method for image synthesis virtually since their introduction. Denoising Diffusion Probabilistic Models are recently setting a new baseline, with remarkable results in Text-to-Image synthesis, among other fields. Aside its usefulness per se, it can also be particularly relevant as a tool for data augmentation to aid training models for other document image processing tasks. In this work, we present a latent diffusion-based method for styled text-to-text-content-image generation on word-level. Our proposed method is able to generate realistic word image samples from different writer styles, by using class index styles and text content prompts without the need of adversarial training, writer recognition, or text recognition. We gauge system performance with the Fréchet Inception Distance, writer recognition accuracy, and writer retrieval. We show that the proposed model produces samples that are aesthetically pleasing, help boosting text recognition performance, and get similar writer retrieval score as real data.

sted, utgiver, år, opplag, sider
Springer, 2023
Serie
Lecture Notes in Computer Science (LNCS), ISSN 0302-9743, E-ISSN 1611-3349 ; 14188
Emneord
Diffusion Models, Synthetic Image Generation, Text Content Generation, Handwriting Generation, Data Augmentation, Handwriting Text Recognition
HSV kategori
Forskningsprogram
Maskininlärning
Identifikatorer
urn:nbn:se:ltu:diva-96364 (URN)10.1007/978-3-031-41679-8_22 (DOI)001346405600022 ()2-s2.0-85173585071 (Scopus ID)
Konferanse
17th International Conference on Document Analysis and Recognition (ICDR2023), San José, CA, USA, August 21-26. 2023
Merknad

ISBN for host publication: 978-3-031-41678-1 (print), 978-3-031-41679-8 (electronic);

This paper has has previously appeared as a manuscript in a thesis.

Tilgjengelig fra: 2023-04-11 Laget: 2023-04-11 Sist oppdatert: 2025-10-21bibliografisk kontrollert
Organisasjoner
Identifikatorer
ORCID-id: ORCID iD iconorcid.org/0000-0002-9332-3188