Exploring the Capabilities of GPT4-Vision as OCR Engine

Many museums and libraries conducted efforts to digitize their assets, and many historic documents are now available as digital images. However, these documents are not directly accessible to retrieval systems that rely on written text and not images. In this study, the novel GPT4-Vision is being studied for its ability of optical character recognition (OCR), in cases where established methods, such as Tesseract may have difficulties. We find that GPT4-Vision provides excellent results even in cases where even humans struggle. We also identified a number of key limitations, including the long runtime implying high energy requirements, the lack of handling of rotated images, the necessity for layout hints, and limitations regarding image size. Even with these limitations, it is expected that large language models and vision transformers will play an important role to make historical documents more accessible for further processing, or directly to users.