Ecosystem overview for everything related to program-synthesis.
ProgramBench is a benchmark developed by facebookresearch designed to evaluate the capability of Language Models (LLMs) to rebuild programs from scratch. It challenges AI agents to architect and implement a complete codebase that reproduces the original program's behavior, given only a compiled binary and its documentation. This tool is crucial for assessing LLMs' performance in reverse engineering and code generation tasks.