Docling from IBM | Open Source Library To Make Documents AI Ready | LlamaIndex

Introduction to Dock Link

Overview of Dock Link

• Dock Link is an open-source AI tool developed by IBM for document parsing and exporting to various formats efficiently.

• The primary focus of the video is on PDF files, but Dock Link can also parse DOCX, PPTX, Excel files, images, HTML, and more.

Video Structure

• The presenter will discuss a blog post detailing different document parsing tools before demonstrating how to set up the environment for Dock Link.

• A key feature includes parsing scanned PDFs and images using simple commands. The integration with Llama Index will be shown for querying parsed documents.

Document Parsing Techniques

Blog Post Reference

• The presenter references a recent Medium post that outlines various PDF parsing techniques including free and open-source options like Dock Link.

• Other tools mentioned include Onstructure (with both free and paid versions), and Llama Pars from Llama Index which offers limited free credits.

Tool Comparison

• A list of various document parsing tools is provided in the blog post for viewers to explore further.

Exploring Dock Link Features

Website Navigation

• The main features of Dock Link are highlighted on its website, including support for multiple file types and easy integration with frameworks like Llama Index and Lang Chain.

Advanced Capabilities

• Upcoming features include equation extraction, code extraction, metadata extraction (title, author), and native Lang Chain extensions.

Installation Insights

Installation Challenges

• Viewers are directed to the GitHub page where installation instructions are available; however, system dependencies may pose challenges during local setup.

Performance Considerations

• Users should be aware that advanced document processing requires powerful hardware; performance may vary based on system capabilities.

Usage Examples

Basic Usage Instructions

• Simple command examples demonstrate how to parse PDFs into Markdown format effectively within just five lines of code.

Additional Resources

• For deeper understanding or application development using Dock Link, viewers are encouraged to explore architectural concepts and integrations listed on the website.

YouTube Stops Repository Overview

Introduction to the YouTube Stops Repository

• The speaker introduces a YouTube stops repository, mentioning that it contains document parsers and plans for future updates with additional parsing tools.

• A recommendation is made to learn about UV (a Python package manager), highlighting its ease of use and efficiency in managing projects.

Setting Up the Environment

• The speaker demonstrates navigating through the project layout, specifically within the document parsers folder.

• It is noted that UV automatically sets up virtual environments when running commands, simplifying the setup process.

Using VS Code and Virtual Environments

• Instructions are provided on how to create a virtual environment using VS Code, emphasizing that users can follow along even without UV.

• The speaker explains how to extract dependencies into a requirements.txt file using a script, making installation straightforward.

Running Commands with UV

Synchronizing Dependencies

• To synchronize dependencies, the command UV sync is introduced; this installs necessary packages without needing manual activation of virtual environments.

• The process of running UV sync is shown, confirming that all required packages are installed seamlessly.

Activating Virtual Environments Manually

• If not using UV, users must manually activate their virtual environment using standard commands like source .VN.

• An example is given on how to run files after activating the environment if not utilizing UV's automated features.

Troubleshooting Installation Issues

Handling Package Installations

• In case of issues with package installations (e.g., testr), instructions are provided for uninstalling and reinstalling correctly.

Exploring Jupyter Notebook Setup

Navigating Documentation and Reports

• The speaker transitions to discussing a Jupyter notebook setup while referencing links to documentation and technical reports related to dock link functionalities.

Data Download Process

• Instructions for downloading data via URL or creating folders for organization are shared.

• A specific example involves downloading a sustainability report from K Company as part of the demonstration.

Final Steps in Project Setup

Reviewing Downloaded Content

• The downloaded sustainability report (84 pages long with tables and figures included) is briefly reviewed as part of setting up project resources.

PDF to Markdown Conversion Process

Overview of the Data Handling

• The speaker discusses simplifying data handling by specifying a source for the PDF and limiting processing to the first 10 pages, avoiding the need to run through all 80+ pages.

Extracting Information from PDF

• The process involves using a tool (p PDF) that is pre-installed during UV sync to extract the first 10 pages of a specified PDF document.

Performance Testing with Specific Pages

• The speaker tests performance by extracting information from page 25, which includes text, pie charts, and bar charts, aiming to evaluate how well the extraction works with single-page data.

Document Conversion Steps

• The conversion process begins with importing documents into a default options setup using a document converter. This step is crucial for parsing and converting content without advanced functionalities.

Output Directory Management

• The output directory is set up using specific path management commands in order to save converted files effectively. A new folder named "par dock" is created for storing these outputs.

Comparing Original PDF and Markdown Output

Visual Comparison of Formats

• A side-by-side comparison shows differences between the original PDF and its markdown version, highlighting how images are noted but not extracted fully in terms of content.

Information Extraction Insights

• The conversion captures headings and key statistics (e.g., percentages), demonstrating effective parsing capabilities while noting limitations regarding image content extraction.

Detailed Examination of Page 25

Content Extraction from Page 25

• Focus on page 25 reveals successful extraction of textual information such as headings related to sustainability strategies while acknowledging that images are only referenced without detailed data extraction.

Unicode Handling in Extracted Data

• Discussion on unicode representation indicates that while some visual elements like charts cannot be processed fully, their presence is acknowledged within the markdown output.

Decoding Unicode Information

Functionality for Decoding Unicode

• A Python function is mentioned that can decode unicode representations found in extracted data, enhancing usability by allowing further manipulation or analysis of this information.

Understanding PDF to Markdown Conversion and OCR Options

Overview of Carbon Footprint Extraction

• The carbon footprint results are influenced by product installation locations, highlighting the importance of context in data extraction.

• Information extraction from PDFs can include encoding elements that cannot be directly extracted as text.

Advanced PDF to Markdown Features

• Customization options for output directories and image resolution settings are available, enhancing user control over the conversion process.

• Various pipeline options exist, including PDF pipelines, table formatters, and OCR functionalities; users should refer to documentation for specific needs.

Utilizing Pipeline Options

• Users must import necessary modules before utilizing pipeline options effectively; this includes understanding default parameters.

• Running commands with two question marks at the end allows users to view default parameters for better customization.

Exploring Documentation and Language Settings

• Reading documentation is crucial for understanding language settings in OCR processes; different languages may require specific configurations.

• Users can experiment with various OCR options like Tesseract's accuracy mode for improved table structure recognition.

Troubleshooting Installation Issues

• Errors related to incorrect installations (e.g., Tesseract OCR not installed properly) can hinder functionality; troubleshooting steps involve checking dependencies.

• If installation fails due to missing dependencies, users should follow specific commands provided in documentation to resolve issues effectively.

Finalizing Installation and Running Commands

• After resolving installation issues, running commands successfully demonstrates the effectiveness of tools used in information extraction.

• Understanding underlying processes during extraction is essential as multiple components (like GPU/CPU usage) interact behind the scenes.

Advanced Document Processing Techniques

Extracting Information from Documents

• The discussion begins with the introduction of a document referred to as "Parts doc Advanced," highlighting its features, including the extraction of specific data such as carbon footprint results.

• It is noted that while some images were not extracted initially, the system identifies them as images and provides information about their content, specifically on page 25 regarding material content.

• The speaker emphasizes that although certain details are still missing in the extraction process, significant information about materials like plastics and organic materials is being captured.

Enhancing Image Extraction

• A new parameter for image mode is introduced to enhance extraction capabilities. This mode embeds images within a base64 format for better integration into documents.

• After implementing this change, the extracted information now includes encoded image data rather than just labeling it as an image, showcasing improved functionality.

• The speaker demonstrates how to decode base64 images online, illustrating how users can retrieve visual data effectively from the processed documents.

Referencing Images in Markdown

• To further improve document processing, a method for referencing images instead of embedding them directly is discussed. This requires minimal changes to existing pipelines.

• By uncommenting a specific line in the code, referenced images are stored in a designated folder within "Parts doc Advanced," allowing for organized access to visual elements.

• The output shows that all relevant information and referenced images are successfully extracted into markdown format, demonstrating effective conversion from PDF.

Chunking Functionality

• An advanced chunking feature is introduced which allows for hybrid chunking of data. This capability enhances how information is parsed and organized during processing.

• The speaker mentions using a specific tokenizer (b a a i BG small e and v15), which aids in efficient parsing and ensures accurate data handling during initial runs.

Command Line Operations

• Transitioning to command line usage, the speaker explains how commands can be executed either through VS Code or terminal interfaces for flexibility in operation.

• A demonstration follows where checks are made to confirm if necessary software (tester OCR version 2.7.1) is installed before running commands related to document processing.

• Finally, there’s an explanation on executing commands with proper syntax to convert pages from PDFs into markdown files seamlessly; however, issues arise when input files do not exist or need adjustments.

Extracting Information from Images and PDFs

Overview of Extraction Capabilities

• The speaker discusses the ability to extract information from images, specifically mentioning a PNG file containing a train schedule.

• Demonstrates running a tool that extracts data from the PNG file without additional parameters, aiming to save it in markdown format.

Accuracy of Extracted Information

• The extraction process is shown to be inaccurate; not all information is captured correctly from the image.

• The speaker notes that while some data is extracted, it may not be complete or accurate, suggesting potential missing parameters for better results.

PDF Extraction Process

• The speaker explains how to extract pages from a PDF by providing its URL, indicating that this process will also convert content into markdown format.

• After processing, errors and warnings are displayed; however, the extraction completes successfully with pages saved in markdown.

Scanned Document Challenges

• A demonstration of passing scanned documents through OCR (Optical Character Recognition) reveals limitations in extracting handwritten text accurately.

• The speaker shares an example of scanning handwritten notes but finds that the extraction yields incorrect or random outputs.

Exporting Figures and Tables

• Discussion on exporting figures indicates that the tool can identify and save images based on their references within documents.

• The process for exporting tables is similar; code snippets are used to create folders where extracted images and tables are stored.

Using MPS for Data Extraction

Overview of MPS and Data Extraction

• The use of MPS (Metal Performance Shaders) allows GPU utilization for efficient data extraction, presenting information in a structured format such as CSV.

• An index is created from 0 to 34, showcasing various tables like "America" and "Europe," which can also be displayed in HTML format.

Integration with Llama Index

• To work with Llama models, it is essential to have the AMA (Advanced Model Architecture) installed and running; previous videos provide guidance on this setup.

• The latest Llama model (3.18 or newer) is utilized alongside Hugging Face embeddings for enhanced performance.

Configuration and Document Loading

• Users can choose between using AMA or Hugging Face models based on preference; the process involves loading pre-trained sentence transformers.

• Ensure that the local host service (Local Host 11:1434) is operational to utilize AMA motors effectively.

Document Processing Pipeline

• Global settings are defined to specify which embedding models and language models (LLMs) should be used instead of default options.

• The pipeline includes importing necessary components from Lama Index, such as core storage context and vector store index.

Extracting Notes from Documents

• A markdown node parser is set up to load data from documents, specifically targeting the K Sustainability Report 2023 for question-answering capabilities.

• After processing, document lengths are confirmed, indicating successful data extraction into a usable format.

Creating an Index for Questioning

Note Creation Process

• Nodes are generated from documents similar to chunks of data; metadata associated with each note can be printed for review.

• A total of 119 nodes were created during this process, allowing detailed insights into the extracted content.

Index Creation and Querying

• An index must be established using a vector store index derived from the processed documents before querying can occur.

• Example queries demonstrate how many countries K operates in; results indicate operations in over 60 countries based on extracted information.

Conclusion on Information Retrieval

• The system provides sources for information retrieval, enabling users to ask multiple questions while tracking their origins within the document.

K Operations and Waste Management Insights

Overview of K's Global Operations

• K operates in over 60 countries, showcasing its extensive global reach.

• The share of landfill waste from K's manufacturing units is reported at 0.6%, indicating a focus on waste management.

Material Composition and Recycling Efforts

• While specific data on plastic and robust content in Monospace 700 was not directly available, it was noted that up to 90% of materials used in elevators can be removed for recycling.

• The question regarding the ways to win areas according to K's strategy highlighted four key approaches: empowered people, marketing and sales renewal, Lin K, and a digital plus physical enterprise model.

Leadership Information

• Philippe Delor is identified as the President and CEO of K, confirming leadership details within the organization.

Conclusion on Data Quality for LLMs

• Emphasized the importance of quality data input for effective outputs from language models (LLMs), reinforcing the principle that "garbage in, garbage out."