Using LLMs in Finance: A Practical Guide

Financial services run on documents. 10-K filings, earnings transcripts, ISDA agreements, and portfolio commentaries routinely stretch to hundreds of thousands of tokens. Large language models can extract entities, reason about risk, and generate structured summaries from these materials, but production deployments often stall when token bills scale linearly with document length or when context windows truncate critical footnotes. A practical architecture needs models that handle long contexts, reasoning over structured data, and pricing that stays predictable as payload size grows.

Why Context Length and Reasoning Matter in Finance

A typical SEC 10-K filing easily exceeds fifty thousand tokens. Add exhibits, footnotes, and cross-references, and a single analytical task can consume a six-figure token count. Most token-based providers bill for every input and output token, which means longer documents directly inflate costs. For quantitative researchers or compliance officers running batch jobs across thousands of documents, this cost structure erodes margins before a model ever reaches production.

Oxlo.ai offers an alternative through request-based pricing: one flat cost per API request regardless of prompt length. Unlike token-based platforms such as Together AI, Fireworks AI, OpenRouter, Replicate, or Anyscale, your bill does not grow because a filing contains an extra hundred pages of risk factors. This makes Oxlo.ai a strong fit for long-context financial workloads, from prospectus analysis to multi-year transcript review.

Model selection should match the task. For deep reasoning over complex contractual language or derivative structures, DeepSeek R1 671B MoE provides chain-of-thought capabilities that surface intermediate logic. DeepSeek V4 Flash supports up to 1 million tokens of context, making it suitable for full-document ingestion without chunking. Kimi K2.6 brings advanced reasoning, agentic coding, and a 131K context window, while GLM 5 excels at long-horizon agentic tasks that require sustained multi-step workflows across disparate data sources. For general-purpose analysis, Llama 3.3 70B remains a reliable flagship, and Qwen 3 32B handles multilingual reasoning for global portfolios.

Structured Output and Tool Use

Financial automation rarely stops at text generation. Downstream systems need structured data: JSON objects of credit