I don't know why Pierre's post is dead, but I wanted to ask if it is accurate to describe Pixelagent as an MCP-compatible memory layer, and what its competitors are.
If memory is the centerpiece, I suggest leading with it, rather than the ambiguous "Your AI Data Infrastructure".
Pixelagent is a reference implementation for a multimodal agent framework to show that an agent class is easy to build and users should be empowered to build their own from scratch for their use cases.
Regarding Memory, to me it's just about Data Storage, Indexing, Orchestration, and Retrieval and I don't know why we should abstract Memory away from users. Memory will mean so many different things for many use cases.
Let's say you want:
- Working memory: Holds current context and immediate interaction history within the agent's context window -> this is just about passing Q&A pairs to maintain context alongside with roles.
- Episodic memory: Stores specific past experiences and interactions -> this is just about indexing past exchanges and having semantic search on it.
- Semantic memory: Organizes specific knowledge in structured formats -> this is just about building a custom logic (udf) to decide how and what to extract insight from and then retrieve it.
Your readme doesn't mention any example of multiple agents. We don't need another wrapper to anthropic/openai/*. Without being able to select between multiple tools/agents - you only need 2 lines of code to achieve llm calls, and another 2 lines for state management.
That said the interface is more idiomatic that others agent frameworks that show up here.
It also wasn't clear where the last N messages is being persisted. Db/file/adapters/?
The goal of this reference agent SDK is to showcase the flexibility of Pixeltable (the underlying unified storage and orchestration system) that is Open Source and Apache 2.0.
Exciting to see a fellow builder in the space! Love how memory management and observability is built-in.
Today, I just re-implemented Google's AlphaEvolve for Perlin noises using my own minimal agentic framework (https://toolkami.com/alphaevolve-toolkami-style/), will probably steal a trick or two from you.
I like PocketFlow. You beat me on the # of lines of code! But does it provide parallelization, caching, orchestration, versioning, observability, lineage, multi-modal support?
As you just showed, building an agent SDK is easy, so what's interesting to me is tackling:
- Infrastructure Sprawl: Juggling separate systems for vector search, state tracking, multimodal data handling, and monitoring leads to fragmented workflows and high operational costs.
- State Management Nightmares: Reliably tracking agent memory, tool calls, and intermediate states across potentially long-running, asynchronous tasks is incredibly difficult.
- Multimodal Integration Pain: Integrating and processing images, audio, video, and documents alongside text requires specialized, often disparate, tooling.
- Observability Gaps: Understanding why an agent made a decision or failed requires visibility into its state and data lineage, which is often lacking.
And doing all of that while finding the right abstraction layer to leave all the application and business logic to the dev/users so they don't feel limited. It's difficult!
Besides, I don't know where you see a commercial offering? Everything is Apache 2.0/Open Source from A to Z.
PocketFlow is not from me, but just my current favorite ;)
I just got the feeling that the lib is tied to pixeltable, but maybe I misunderstood? Maybe that's why this is dead?
pocketflow is completely standalone and the main thing is that you vibe code what you need (works awesome so far!).
I don't want to sideline the discussion about pixelagent, but here's some more about pf:
It all started with https://github.com/pixeltable/pixeltable which is the core project we are actively working on and the problem space of working with video and frames for Computer Vision teams who struggle with the explosion of data and maintaining lineage and versioning for frames of video.
The same problems arise with LLMs for Documents and Chunks, Audio and Samples, etc.
LLM is just a function call. We are agnostic to whatever framework, library, and data format are used; we are focused on solving the data plumbing issues (mulitmodal storage and orchestration) with a simple python sdk: https://github.com/pixeltable/pixeltable.
It really depends on the use case. For Pixelagent, I don't really look at competition as mentioned by the fact that this is a reference implementation.
People should build their own; it's easy. What's difficult is the underlying data infrastructure (storage and orchestration).
I don't know why Pierre's post is dead, but I wanted to ask if it is accurate to describe Pixelagent as an MCP-compatible memory layer, and what its competitors are.
If memory is the centerpiece, I suggest leading with it, rather than the ambiguous "Your AI Data Infrastructure".
You can indeed turn anything that you want into an MCP server, e.g. https://github.com/pixeltable/pixeltable-mcp-server.
Pixelagent is a reference implementation for a multimodal agent framework to show that an agent class is easy to build and users should be empowered to build their own from scratch for their use cases.
Regarding Memory, to me it's just about Data Storage, Indexing, Orchestration, and Retrieval and I don't know why we should abstract Memory away from users. Memory will mean so many different things for many use cases.
Let's say you want:
- Working memory: Holds current context and immediate interaction history within the agent's context window -> this is just about passing Q&A pairs to maintain context alongside with roles.
- Episodic memory: Stores specific past experiences and interactions -> this is just about indexing past exchanges and having semantic search on it.
- Semantic memory: Organizes specific knowledge in structured formats -> this is just about building a custom logic (udf) to decide how and what to extract insight from and then retrieve it.
I've implemented them all in this example: https://github.com/pixeltable/pixelbot
Your readme doesn't mention any example of multiple agents. We don't need another wrapper to anthropic/openai/*. Without being able to select between multiple tools/agents - you only need 2 lines of code to achieve llm calls, and another 2 lines for state management.
That said the interface is more idiomatic that others agent frameworks that show up here.
It also wasn't clear where the last N messages is being persisted. Db/file/adapters/?
Good luck!
Thanks!
https://github.com/pixeltable/pixelagent/tree/main/examples/... for multiple tools/agents. A tool is just a UDF. You can have as many as you want.
The goal of this reference agent SDK is to showcase the flexibility of Pixeltable (the underlying unified storage and orchestration system) that is Open Source and Apache 2.0.
This is where storage is defined: https://github.com/pixeltable/pixelagent/blob/main/pixelagen.... These are very simple examples.
Here's how an Agentic Reddit bot would work for instance: https://github.com/pixeltable/pixeltable/tree/main/docs/samp...
Exciting to see a fellow builder in the space! Love how memory management and observability is built-in.
Today, I just re-implemented Google's AlphaEvolve for Perlin noises using my own minimal agentic framework (https://toolkami.com/alphaevolve-toolkami-style/), will probably steal a trick or two from you.
Congrats! Take whatever you want :)
Yes: Memory, observability, versioning, and lineage being built-in is a derivative of unifying orchestration and storage: https://docs.pixeltable.com/docs/datastore/computed-columns.
Pixelagent: Agent Engineering Blueprint https://www.youtube.com/watch?v=BS6PRsnxkBA
I'll just leave this here: https://github.com/The-Pocket/PocketFlow 100 lines of code agent lib that's not tied to a commercial offering
I like PocketFlow. You beat me on the # of lines of code! But does it provide parallelization, caching, orchestration, versioning, observability, lineage, multi-modal support?
As you just showed, building an agent SDK is easy, so what's interesting to me is tackling:
- Infrastructure Sprawl: Juggling separate systems for vector search, state tracking, multimodal data handling, and monitoring leads to fragmented workflows and high operational costs. - State Management Nightmares: Reliably tracking agent memory, tool calls, and intermediate states across potentially long-running, asynchronous tasks is incredibly difficult. - Multimodal Integration Pain: Integrating and processing images, audio, video, and documents alongside text requires specialized, often disparate, tooling. - Observability Gaps: Understanding why an agent made a decision or failed requires visibility into its state and data lineage, which is often lacking.
And doing all of that while finding the right abstraction layer to leave all the application and business logic to the dev/users so they don't feel limited. It's difficult!
Besides, I don't know where you see a commercial offering? Everything is Apache 2.0/Open Source from A to Z.
PocketFlow is not from me, but just my current favorite ;)
I just got the feeling that the lib is tied to pixeltable, but maybe I misunderstood? Maybe that's why this is dead? pocketflow is completely standalone and the main thing is that you vibe code what you need (works awesome so far!).
I don't want to sideline the discussion about pixelagent, but here's some more about pf:
- https://the-pocket.github.io/PocketFlow/design_pattern/multi... (multi agent, queue) - https://github.com/The-Pocket/PocketFlow/tree/main/cookbook/ here are more advanced examples. Pretty easy to follow imho.
PS: re the observability, yesterday I coded tracing for pocketflow, just need to put it up on github haha
[dead]
Can I ask, does this have anything to do with pixels or is it just a name?
It all started with https://github.com/pixeltable/pixeltable which is the core project we are actively working on and the problem space of working with video and frames for Computer Vision teams who struggle with the explosion of data and maintaining lineage and versioning for frames of video.
The same problems arise with LLMs for Documents and Chunks, Audio and Samples, etc.
If you are into that:
- Object Detection on Video Frames: https://github.com/pixeltable/pixeltable/blob/main/docs/note...
- Text and images similarity search: https://github.com/pixeltable/pixeltable/tree/main/docs/samp...
LLM is just a function call. We are agnostic to whatever framework, library, and data format are used; we are focused on solving the data plumbing issues (mulitmodal storage and orchestration) with a simple python sdk: https://github.com/pixeltable/pixeltable.
Congrats. What tools do you see as the closest alternatives and how do they compare?
It really depends on the use case. For Pixelagent, I don't really look at competition as mentioned by the fact that this is a reference implementation.
People should build their own; it's easy. What's difficult is the underlying data infrastructure (storage and orchestration).