I built a 17-agent AI swarm on my phone — here's how

I built a 17-agent AI swarm on my phone — here's how

Okay, buckle up. This is a bit of a wild ride. I recently accomplished something I wasn't even sure was possible: I built and deployed a 17-agent AI swarm that runs entirely on my phone. No cloud connection, no server – just the processing power of a mid-range Android device. It’s not solving world hunger (yet), but it's a fascinating exploration of on-device AI, agent-based modeling, and the surprisingly capable hardware we carry in our pockets.

The core idea was to build a system where individual AI 'agents' could collaborate to achieve a common goal, but do so without centralized control or constant communication. Think of a flock of birds or a school of fish – decentralized intelligence achieving complex behaviors. The application I chose to showcase this was a simple foraging simulation: agents need to ‘find’ and ‘collect’ virtual resources scattered across a simulated 2D plane.

Why do this?

Several reasons. Firstly, I'm a huge proponent of edge computing. Moving processing to the device reduces latency, improves privacy (no data sent to the cloud), and can even allow for offline functionality. Secondly, the advancements in mobile chipsets – specifically Neural Processing Units (NPUs) – are seriously impressive. And finally, let's be honest, it’s a cool challenge!

The Tech Stack

This wasn't a smooth ride. I went through a few iterations before settling on a workable stack:

• Language: Python – It’s still the king of rapid prototyping for AI.
• Framework: Kivy – This cross-platform Python framework allows me to build a touch-friendly UI and handle the graphical rendering. It's not the fastest, but it's incredibly accessible.
• AI Engine: TensorFlow Lite – Crucial for running the models on the phone’s NPU. TensorFlow Lite significantly optimizes models for mobile and embedded devices.
• Agent Logic: Custom implementation using simple neural networks. I wanted to keep it relatively lightweight.
• Device: A Google Pixel 6a – Not top-of-the-line, but capable enough.

The Agent Architecture

Each agent is governed by a small neural network that takes sensory inputs and outputs actions. The inputs are:

• Distance to nearest resource: A normalized distance value.
• Direction to nearest resource: Angle relative to the agent's heading.
• Agent Density: Number of other agents within a certain radius.
• Energy Level: A value representing the agent’s remaining ‘energy’.

The outputs are:

• Movement Angle: Direction to move (0-360 degrees).
• Movement Speed: How fast to move.
• Collection Action: Whether to attempt to collect a resource (binary).

Here's a simplified representation of the neural network architecture (using Keras within Python):

import tensorflow as tf

model = tf.keras.models.Sequential([
  tf.keras.layers.Dense(64, activation='relu', input_shape=(4,)), # 4 inputs
  tf.keras.layers.Dense(32, activation='relu'),
  tf.keras.layers.Dense(3, activation='sigmoid')  # 3 outputs (angle, speed, collect)
])

model.compile(optimizer='adam',
              loss='mse', # Mean Squared Error - appropriate for regression
              metrics=['accuracy'])

This network is very basic. We’re not talking GPT-level intelligence here. But it’s sufficient for the task and, crucially, lightweight enough to run efficiently on a phone.

Training the Swarm

Training 17 of these networks on the phone was a non-starter. The Pixel 6a doesn't have the horsepower to handle that. So, I trained a single ‘master’ network using a more powerful machine, then cloned its weights to the 17 agents. The training environment consisted of:

• Reward System: Agents received positive rewards for collecting resources and negative rewards for collisions (with each other or the environment boundaries) and running out of energy.
• Genetic Algorithm: To introduce some variation between agents, I applied a subtle genetic algorithm. After training the master network, I introduced small random mutations to the weights of each agent’s network, creating slight differences in their behavior.

Deployment and Optimization

This is where things got really interesting. Getting TensorFlow Lite to play nicely with Kivy required some significant tweaking. Key steps included:

• Model Conversion: Converting the Keras model to a TensorFlow Lite model using tf.lite.TFLiteConverter.
• Quantization: Applying post-training quantization to reduce the model size and improve inference speed. I experimented with different quantization levels, finally settling on 8-bit integer quantization for a good balance between accuracy and performance.
• Interpreter Setup: Loading the TFLite model using the TensorFlow Lite Python interpreter within the Kivy app.

import tflite_runtime.interpreter as tflite

interpreter = tflite.Interpreter(model_path="agent_model.tflite")
interpreter.allocate_tensors()

• Input/Output Handling: Transforming the sensory data into a format the model expects (a NumPy array) and interpreting the output.

The Challenges (and How I Overcame Them)

• Performance Bottlenecks: Running 17 neural networks simultaneously on a phone is resource-intensive. Profiling revealed that the most significant bottleneck was the model inference time. Quantization helped, but I also implemented a simple frame skipping mechanism: only running the AI calculations every few frames to maintain a playable framerate.
• Collision Detection: Collision detection between agents was surprisingly costly. I simplified it by using a bounding box approach and only checking for collisions between agents within a certain proximity.
• UI Responsiveness: Kivy's UI felt sluggish with all the AI calculations running in the background. I offloaded some UI updates to separate threads to avoid blocking the main thread.

The Result?

The simulation isn’t photorealistic. It’s deliberately simple – just colored circles representing agents and squares representing resources. But it works. You can see the agents foraging, bumping into each other, and collectively converging on the resources. The variation in their behaviors (introduced by the genetic algorithm) leads to emergent swarm dynamics that are quite fascinating to watch.

The swarm isn't perfect. Sometimes, agents get stuck in loops. Sometimes, they exhibit chaotic behavior. But it demonstrates the feasibility of running a complex, multi-agent system entirely on a mobile device. It's a testament to the power of on-device AI and the impressive capabilities of modern mobile chipsets.

Future Directions

I’ve only scratched the surface. Future improvements could include:

• Reinforcement Learning on Device: Exploring