My Battle with AI: Getting Consistent Docstrings in Legacy Python

Inheriting a monster of a Python codebase is a rite of passage for many indie devs, myself included. This particular beast, built with Python 3.8 a few years back, was a labyrinth of uncommented functions and cryptic one-liners. My first thought? "AI, you beautiful thing, this is your moment!" I envisioned feeding it the entire repo and having beautifully formatted, consistent docstrings spit out like magic.

Spoiler: It wasn't magic. It was more like wrestling a greased pig in the dark.

My goal was simple: get Google-style docstrings for every function, class, and method. Not just any docstrings, but accurate ones that actually explained what the code did, its parameters, and what it returned, following a consistent style guide. I needed this for future me, and any poor soul who might join the project later.

Step 1: The AI Dream – Just Generate It!

My initial approach was naive optimism. I opened up my VS Code, fired up GitHub Copilot, and tried a few prompts with ChatGPT 4o. "Generate a Google-style docstring for this function:" followed by the function. Simple, right?

Take this example function from the codebase:

python
def calculate_invoice_total(items, discount_code=None):
total = sum(item['price'] * item['quantity'] for item in items)
if discount_code == 'SAVE10':
total *= 0.9
elif discount_code == 'FREESHIP':
# Shipping logic is elsewhere, this just flags it
pass
return total

Copilot would often give me something like this, which isn't terrible but misses context:

python
def calculate_invoice_total(items, discount_code=None):
"""Calculates the total price of an invoice.

Args:
    items (list): A list of items, each with 'price' and 'quantity'.
    discount_code (str, optional): A discount code to apply. Defaults to None.

Returns:
    float: The total price after applying discounts.
"""
# ... rest of code

It's okay, but it doesn't mention which discount codes are recognized or the specific logic for them. It's generic, which is fine for a quick start, but not for accurate legacy documentation.

Step 2: The Reality Check – Generic Blah and Outright Lies (The Step That Didn't Go As Planned)

This is where my AI dream hit the wall. When I fed it more complex functions, or functions with obscure business logic, the AI would hallucinate. It'd invent parameters that didn't exist, claim a function returned a boolean when it actually returned a dictionary, or completely misinterpret the purpose of a tricky conditional.

Here was a common pattern: I'd have a function like process_user_data(user_id, config) where config was a deep, nested dictionary passed around the system, often missing keys if not handled correctly. The AI would often describe config as dict: User configuration settings. which is technically true but utterly unhelpful for knowing which settings were expected or required. It'd gloss over the specific keys it needed or the structure.

I probably spent a solid 3 hours trying to get it to "just do it" – trying different prompts, asking it to "be more specific," "analyze the code carefully" – but the output was inconsistent. One docstring would be verbose, the next sparse. It was generating a docstring, but not my docstring, nor an accurate one I could trust without a full line-by-line review.

Step 3: The Hybrid Approach – Human Input, AI Refinement

I realized AI wasn't going to be the magic bullet for creation in complex, unfamiliar code. Its strength, I discovered, was in refinement and consistency once given a solid starting point.

My new process evolved into this:

1. I write the core idea: For each function, I'd first jot down a rough, 1-2 sentence description of what it actually does, based on reading the code and understanding the business logic. I'd manually identify the parameters and what they represented. This initial step took me maybe 30 seconds to a minute per function.

   python
   def get_order_details(order_id, user_permissions):
   # My rough thought: This fetches an order by ID, checking if the user can see it.
   # ... actual code ...

2. AI for Structure & Expansion: Then I'd feed this function, along with my rough description (or sometimes just the function itself after my mental pass), to ChatGPT 4o with a specific prompt: "Generate a Google-style docstring for this Python function. Focus on accurately detailing parameters, return values, and potential exceptions. Ensure consistency with standard Google docstring format." I often included a paragraph about the specific context of user_permissions or config if it was tricky.

   The AI, now guided by my initial understanding or a clearer prompt, would produce something much more robust:

   python
   def get_order_details(order_id, user_permissions):
   """Fetches the details of a specific order, verifying user permissions.

   This function retrieves an order record from the database based on its ID.
   It checks the user_permissions dictionary to ensure the requesting user
   has the necessary 'read_orders' permission and that they are either the
   owner of the order or an admin.
   
   Args:
       order_id (str): The unique identifier of the order.
       user_permissions (dict): A dictionary containing the user's roles and
           permissions, e.g., {'roles': ['admin'], 'user_id': 'abc'}.
   
   Returns:
       dict: A dictionary containing the order details if found and authorized.
   
   Raises:
       PermissionError: If the user does not have adequate permissions to view the order.
       ValueError: If order_id is invalid or not found.
   """
   # ... actual code ...
   

3. Human Review & Correction: This is crucial. I'd then meticulously review the AI-generated docstring. I'd check parameter types, return values, and ensure the description matched the actual code logic. I'd often catch small discrepancies – an int instead of str, a forgotten None return case. This step was much faster than writing it from scratch, or endlessly debugging the AI's full-auto mode.

This hybrid method cut down my docstring generation time significantly, probably by about 60-70% compared to doing it all manually, and produced far more consistent and accurate results than letting the AI run wild. It's not a silver bullet, but it's a hell of a force multiplier when you're wading through a legacy codebase.

AI is an incredible assistant, but for tasks requiring deep contextual understanding and absolute accuracy, especially in unfamiliar code, the human is still firmly in the loop. It just makes the loop a whole lot shorter. 💪