What is it?¶
DataFrameIt processes text in DataFrames using Large Language Models (LLMs) and extracts structured information defined by Pydantic models. One function, one model and one prompt are all you need.
from pydantic import BaseModel
from typing import Literal
import pandas as pd
from dataframeit import dataframeit
class Sentiment(BaseModel):
sentiment: Literal['positive', 'negative', 'neutral']
confidence: Literal['high', 'medium', 'low']
df = pd.DataFrame({'text': ['Excellent product!', 'Terrible service.']})
result = dataframeit(df, Sentiment, "Analyze the sentiment of the text.")
Features¶
Multiple Providers¶
OpenAI, Google Gemini, Anthropic, Groq and others via LangChain, plus Codex and Claude Code through the official SDKs.
Structured Output¶
Automatic validation with Pydantic. Define fields, types, and descriptions; a response that does not match the model goes back to the LLM with the error for correction.
Resilience¶
Automatic retry with exponential backoff, configurable rate limiting and periodic checkpoints to resume long runs.
Performance¶
Parallel processing that reduces the workers on rate limits. With parallel_requests > 1, the summary at the end of the run includes throughput metrics.
Web Search¶
Tavily or Exa to enrich data with information from the internet, with per-field search and conditional fields.
Multiple Inputs¶
pandas or Polars DataFrame and Series, list and dictionary. The output is a DataFrame, a Polars one when the input is Polars.
Quick Installation¶
pip install dataframeit[openai] # OpenAI (default provider)
pip install dataframeit[google] # Google Gemini
pip install dataframeit[anthropic] # Anthropic
pip install dataframeit[all] # providers with an extra except Codex, plus web search, Polars and Excel