Using the output with LLMs and search (RAG)

Language models and search indexes work best on clean text that keeps its structure. DocSlice gives you both the text and, when you need it, just the facts.

Recipe: documents to a search index

  1. Convert to Markdown with format=markdown. Headings, lists and tables survive, which helps both retrieval and the model's reading. Scans and photos are read with OCR first (see Scanned documents and images).
  2. Split into chunks along the headings, and cut any section that is still too long.
  3. Index each chunk with your embedding model, together with the file name and the heading it came from.
  4. At question time, retrieve the best chunks and give them to the model with the question.

A small splitter for the second step:

import re


def chunks(markdown, max_chars=2000):
    """Splits a Markdown document at its second-level headings, then cuts any section that is still too long."""
    pieces = []
    for section in re.split(r"(?m)^(?=## )", markdown):
        section = section.strip()
        while len(section) > max_chars:
            cut = section.rfind("\n\n", 0, max_chars)
            cut = cut if cut > 0 else max_chars
            pieces.append(section[:cut].strip())
            section = section[cut:].strip()
        if section:
            pieces.append(section)
    return pieces

Call it on the data of a format=markdown conversion.

Recipe: extract the facts, not the text

When the model needs only a few values (parties of a contract, a total, a date), ask for them with a schema and pass the small JSON to the prompt instead of the whole document. It is cheaper, faster and easier to check.

Practical advice

  • Mind input_truncated. The AI extractor reads the first 25,000 characters of a document. With markdown there is no such limit on the text you receive, so split long documents yourself.
  • Use batches for many files, or a ZIP archive (see Batch and ZIP), and asynchronous mode for big ones.
  • Keep the source. Store the file name and the chunk's heading next to each vector so an answer can cite where it comes from.
  • Do not keep what you do not need. Converted text is yours to store, but DocSlice deletes uploaded files immediately and asynchronous results after 24 hours.