We are thrilled to announce that our team participated in the recent AI Hackathon organized by AI Engineer Foundation Europe and TeamGPT, and we’re proud to share that we secured 3rd place!

During the hackathon, AI enthusiasts and developers came together to collaborate, innovate, and build cutting-edge solutions.

Our team worked tirelessly to create a powerful AI assistant that helps manage knowledge and streamline daily workflows for both individuals and businesses.

This was an incredible opportunity to showcase our skills and push the boundaries of AI-driven solutions.

A big thank you to all the organizers, judges, and sponsors of the event, including Iliya Valchanov, Michael Montano, TeamGPT, LAUNCHub Ventures, Iris.ai, Appolica | Venture studio - and everyone who made this event possible.

More about the project we built entirely from scratch:
A powerful web-based AI assistant that helps people and businesses manage their knowledge and everyday workflows:
- It allows you to log and organize anything (notes, tasks, calendar) from anywhere, as soon as it pops into your mind using chat, voice, Siri, Google Assistant
- It keeps up with context and fetches anything it needs to complete a task even if the communication channel changes or even if you give it more vague instructions

Check out the Beta version: 🎥👇

[Loom](https://www.loom.com/share/21b7062d75424814ab6d2469ac14e1ad)

🥉 3rd place at AI Hackathon 🥳 Check out what we made in a weekend full of innovation, emotion and fun.

1728405892185

Celebrating our Medal at Sofia AI Hackathon 2024

Generating documents dynamically is a common requirement in many applications, especially in cases like legal forms, contracts, and agreements where personalized user data needs to be inserted into predefined templates. Laravel, combined with external libraries like PhpWord, provides an efficient way to handle this.

### Key Components of Dynamic Document Generation

The process of generating dynamic documents in Laravel generally involves a few key steps:

1. **Template Processing**: The foundation of this approach is using pre-designed templates, usually in formats like `.docx`. These templates contain placeholders that will be replaced with actual user data. PhpWord’s `TemplateProcessor` is often used to load these templates, replace the placeholders, and generate the final document.

2. **Mapping User Data to Placeholders**: The core logic revolves around mapping user-specific information, such as names, addresses, or other relevant details, to the placeholders in the template. This ensures that each document is tailored to the specific user or case at hand.

3. **Dynamic Image Insertion**: In some cases, signatures or other images need to be dynamically inserted into the document. For instance, a contract may require both the client’s and attorney’s signatures to be embedded directly into the final document. This is done by specifying the image paths in the code and replacing the placeholders in the document with actual image files.

4. **Saving the Generated Document**: Once the placeholders are replaced with the appropriate data and any images are inserted, the final step is saving the document to a specified path. This allows it to be downloaded or sent to users via email.

### Common Use Cases

This approach is particularly useful in industries like legal services, where contracts, power of attorney forms, and other formal documents are frequently required. By automating the document generation process, businesses can save time and reduce human error.

### Benefits of Using Laravel for Document Generation

- **Efficiency**: By leveraging templates, you can generate documents quickly without manually drafting them for each user or case.
- **Flexibility**: PhpWord’s support for dynamic placeholders allows you to easily map various types of user input into your document templates.
- **Customization**: With the ability to insert dynamic images like signatures, you can ensure that your documents meet professional standards without requiring additional post-processing.

### Conclusion

Dynamic document generation in Laravel, powered by PhpWord, simplifies the process of creating personalized documents based on user input. Whether for legal contracts, agreements, or other formal documentation, this approach provides a flexible and efficient solution to meet business needs.

Learn the concepts of generating dynamic documents in Laravel using PhpWord. The article covers template processing, inserting user data, and adding dynamic images like signature.

Dynamic Document Creation article image

Dynamically Generating Documents in Laravel

As an engineer at Lexis, I would like to share some helpful bits of my experience solving complex business intelligence problems. Recently, I was assigned a rather daunting task of extracting financial information from over half a million documents of financial reports from various companies. This task, though challenging, allowed me to delve deeper into the applications of the more recent AI advances, mainly through OpenAI embeddings and vector databases, for business intelligence problems. How, you ask? Let's take a journey through the process in my comprehensive case study.

## The problem

We were tasked to extract key-value pairs from inconsistent structure and text from business documents. Therefore, we couldn't quickly develop an algorithm to solve the problem and had to implement a solution to leverage the new advances in LLM. We used a cloud solution to train an AI model to extract the key-value pairs. However, it was still pretty challenging to deal with the sheer volume of documents, as not all documents held relevant information, and if we had to process each page, not only would it take more time, but it would be much more expensive. Manually sifting through these documents would be even more time-consuming and economically unviable. The solution? A classification program that combines OpenAI embeddings and ChromaDB vector databases, using Pytesseract for Optical Character Recognition (OCR). 

Before we delve into how these technologies helped me overcome the challenge, let's understand what vector embeddings and vector databases are. 

### Vector embeddings

In a high-dimensional space, vector embeddings are mathematical representations of objects, such as words or documents, capturing semantic meanings based on their context. These vectors provide a way for algorithms to understand the content and context of documents. 

### Vector databases

Vector databases, on the other hand, are databases designed to store and query these vector embeddings efficiently. They enable us to perform similarity search at scale, which is critical in tasks such as semantic search, relatedness search, or, in our example - classification. 

### Implementation

This is where OpenAI and ChromaDB came into the picture. OpenAI provides a powerful tool to generate embeddings for our documents, while ChromaDB allows us to store and query these embeddings efficiently. By leveraging the power of vector embeddings and vector databases, we can classify documents based on how close their vector representations are, which will help us identify relevant pages in a document.

First, I split the PDF documents into individual pages. For each page, I created a MySQL record, applied OCR using Pytesseract, and generated an embedding using OpenAI. Using ChromaDB, I queried the page embedding with the embeddings of already classified pages (that I organized personally) stored in the vector database. The result of the query would give me the most related page to the one I am querying, and I would classify it with the same type, storing the type back in the database.

This process enabled me to filter out irrelevant pages, leaving only the ones that needed further processing. This streamlined the process and reduced the cost by about 80% of the original estimate.

### A basic example

I will give you a simple example to showcase an implementation of a Python program that does a similar job. To implement this solution, you'll need to create a MySQL database that contains the information for each image and a ChromaDB vector database, which you will use to query the vector embeddings.

Here's a simple schema for the MySQL database:

```sql
CREATE TABLE documents (
    id BIGINT UNSIGNED PRIMARY KEY,
    name VARCHAR(255),
    file_path VARCHAR(255), # path of the document in the filesystem
);

CREATE TABLE document_pages (
    id BIGINT UNSIGNED PRIMARY KEY,
    document_id BIGINT UNSIGNED,
    type ENUM(‘invoice’, 'balance_sheet’, ‘income_sheet’, ‘none), # here we store the type of the image once we classify it
    file_path VARCHAR(255),
    FOREIGN KEY (document_id) REFERENCES documents(id)
);
```

We need to manually classify at least a couple hundred images and save them in a folder. The assigned type of each image should be contained in the name of the image, separated by a dash, for example: ```'[name]-invoice.jpg'``` or ```'[name]-none.jpg'```. Then, we will embed these images into a ChromaDB vector database using the following example:

```python
import chromadb
import glob
import os
import pytesseract as pt
from openai import OpenAI
import uuid

openai = OpenAI('your-api-key')

client = chromadb.PersistentClient(path="chromadb")

# create the classification chromadb collection
classification_collection = client.get_or_create_collection(name="classification_collection",  metadata={"hnsw:space": "cosine"})

classified_images = glob.glob("classified_images/*.jpg") # the location of the folder containing the manually classified images

for classified_image in classified_images:
    # get base name of image
    image_type = os.path.basename(classified_image).split(".")[0].split('-')[1]

    # get the ocr text
    ocr_text = pt.image_to_string(classified_image)

    embedding = openai.embeddings.create(
        input=ocr_text.lower(),
        model="text-embedding-ada-002"
    ).data[0].embedding

    # embed the image in the chromadb database
    classification_collection.add(
        embeddings=embedding,
        metadatas={'type': classified_image.type},
        ids = str(uuid.uuid4())
    )

```
After creating the classification database, you can use the following Python code to perform OCR, generate embeddings, and classify the pages of a given document page:

```python
from models import DocumentPage
import chromadb
import pytesseract as pt
from openai import OpenAI

openai = OpenAI('your-api-key')

client = chromadb.PersistentClient(path="chromadb")

# create the classification chromadb collection
classification_collection = client.get__collection(name="classification_collection")

for DocumentPage in DocumentPage.all():
    # get the ocr text
    ocr_text = pt.image_to_string(DocumentPage.file_path)

    embedding = openai.embeddings.create(
        input=ocr_text.lower(),
        model="text-embedding-ada-002"
    ).data[0].embedding

    # embed the image in the chromadb database
    query = classification_collection.query(
        query_embeddings=embedding,
        n_results=1
    )

    DocumentPage.type = query['metadatas'][0][0]['type']

    DocumentPage.save()
```

## Conclusion

This simple example should illustrate the process of classifying images using the latest trends in LLMs and vector databases. The synergy of OpenAI embeddings and ChromaDB vector databases has revolutionized our approach to document classification, making it more efficient and cost-effective. 

The article has given you a glimpse into the power of AI tools in business intelligence. As we continue to explore and experiment, we are excited about the endless possibilities that AI holds for us, and we will keep sharing our insights about how we can use them in real-world scenarios.


Uncover how OpenAI and ChromaDB revolutionized financial data extraction, streamlining document classification for efficiency.

Unleashing the Power of AI in Business Intelligence: Document Classification using OpenAI and ChromaDB

Lexis Solutions was founded with the sole intent of providing companies with a product that meets their business needs. Having entrepreneurs experience ourselves, we provide ideas, new viewpoints, and useful tricks which ultimately save the client time and money. We replenish our creativeness by actively engaging with the latest technologies, which occasionally bring about a new app.

Our team is excited to announce that Clutch has named Lexis Solutions a top B2B company in Bulgaria in the development category! We are honored to be listed among the top app developers in Bulgaria on Clutch!

Based in Washington, DC, Clutch is a B2B rating and review platform. Their team conducts independent research into the past clients of companies listed on their site. These verified reviews form the basis of their rankings and awards.

> The team is more than happy to be recognized as one of the best in the region by Clutch. We strive to provide our partners with unparalleled engagement and transparency in the work process. Getting the recognition from Clutch only strengthens our belief that this is the right path.
> CEO

We are thankful for all of our wonderful clients, especially those that left us reviews on Clutch.co! Check out what they had to say about their experience working with us:

> They're very flexible, and they always provide real quality. So far, Lexis Solutions hasn't provided us any bad work. They test everything before delivering it, which is outstanding."
> CIO, Technology Startup

Both this award and our perfect 5-star rating were made possible by your support over the years!

Deyan

Clutch's team article about Lexis Solutions


Lexis Solutions Named a Top Bulgarian B2B Company

Build your digital solutions with expert help