Researchers at Carnegie Mellon University created a software company full of AI agents that staffed various roles such as software engineers, project managers, and HR staff. The company AI staff was set day-to-day tasks that mirror a real life company and measured on their progress, using a number of popular AI products. The best performing agents only managed to complete 24% of the tasks, and some cheated to complete tasks such as renaming usernames in the company chat when their intended correspondent didn't respond.
