Business: China's dominance in humanoid robots, AI to go nuclear, Trouble with Synthetic Data, OpenAI's new investment round from Apple, Fed's early access to OpenAI & Anthropic models, Staggering numbers from Meta & OpenAI on LLM usage and adoption

Technology: Paper - Automated Design of Agentic Systems, Google's new RAG framework - Speculative RAG, Experimental results from 4bit quantization of NVIDIA Minitron models

Long Read: Ark-Invest Big Ideas 2024 Annual Report

Resources: A comprehensive guide to LLM technology landscape - A technical paper

AI in Businesses

•  China's domination in human robots. The five-day 2024 World Robot Conference had two dozen humanoid robots from the mainland with Tesla being the only foreign competition (Ben Jiang, South China Morning Post)

• AI to go nuclear? AWS, Microsoft, and Google are going nuclear to build and operate mega data centres better equipped to meet the increasingly hefty demands of generative AI. Several factors are driving this - 1) The training runs need enormous power within a single data centre to shorten the training cycles - nuclear would be justifiable and efficient in this scenario 2) Gives substantial self-sufficiency for hyper scalers and not be subjected to grid level bottlenecks and outages (Paula Rooney, CIO.com)

• When A.I.’s Output Is a Threat to A.I. Itself? The article challenges the notion of increasing training data supply from synthetic data or data on the internet which are already written from LLM-generated content (OpenAI reported that it generates 100 billion words per day, surely some of that is getting published which recycles back). The author challenges the lack of originality with some reasonable illustrations of facial images and digit identification tasks (Aatish Bhatia, NY Times)

• Apple, Nvidia Are in Talks to Invest in OpenAI - somewhat surprising that Apple did not try to create in-house LLM capabilities (this is in addition to using Gemini in their products and plans to incorporate Cohere in some other aspects) - probably the only 1 company not attempting to do so of the Magnificent 7 companies. Do they think that the value will be generated elsewhere - if so they are thinking very differently to Tesla and Meta! (Tom Dotan and Aaron Tilley, The Wall Street Journal)

• Feds to get early access to OpenAI, Anthropic AI to test for doomsday scenarios. Contrary to initial concerns, the AI industry is indeed supporting government scrutiny and involvement in model approvals (Ashley Belanger, ARS Technica)

• Some of the numbers are mindboggling - Meta reports that there are close to 350Million downloads of Llama model series since its inception and OpenAI says its weekly active user base increased from 100 million users last year to 200million users (1 year span) (Engadget & Axios)

  
  

Technology updates from AI

• Can we automate the design of agentic systems rather than relying on manual efforts? This paper describes a novel approach to the Automated Design of Agentic Systems. It is trying to prove the historic precedent of hand-designed systems eventually becoming learned systems or auto-designed systems. Meta Agent Search, an ADAS algorithm, uses a meta-agent (LLM) to iteratively program new agents in code. It builds on previous discoveries stored in an archive, enabling the exploration of any possible agentic system within a Turing-complete search space.

   

  The following efficiency improvements are noted in the paper

   

  ARC logic puzzle: 14% accuracy improvement

  DROP reading comprehension: +13.6/100 F1 score

  MGSM math tasks: +14.4% accuracy

  GSM8K math (transfer): +25.9% accuracy

  GSM-Hard math (transfer): +13.2% accuracy

   

  (University of British Columbia, 2Vector Institute, 3Canada CIFAR AI Chair)

   

• Google Cloud AI team published Speculative RAG is a novel Retrieval Augmented Generation framework that uses a smaller specialist LM to generate draft texts that are then fed to a larger generalist LM to verify and select the best draft. The 'Retriever' responds with multiple relevant documents. The Retriever LM (which instruct fine-tuned already) sends multiple draft responses which Larger 'verifier' LLM will evaluate and select the best draft. Speculative RAG achieves state-of-the-art performance both in accuracy and efficiency (Chen Yu Lee, Zilong Wang, Google Cloud AI)

• NVIDIA published Minitron versions of Llama 3.1 and Mistral-Nemo (using pruning the least important weights, followed by retaining through knowledge distillation. This approach reduces the model's size while preserving accuracy. This article Mistral Nemo: 4.1x Smaller with Quantized Minitron proposes a further reduction by Quantization using AutoRound. Surprisingly the performance on 4 language tasks is still better than the original Llama 3.1 8B model. We can still fine-tune the quantized Minitron with a parameter-efficient fine-tuning method like QLoRA for adaptive usage on new training data - overall an excellent discovery (The Kaitchup, Substack Channel)

Long Read

• Ark-Invest's Big Ideas 2024 report which is the latest in the series of annual 'Big Ideas' report series published by the investment firm - a front runner in investment in disruptive/exponential technologies

• 5 disruptive technology Platforms are - 1) Public Block Chains 2) Multiomic Sequencing 3) Artificial Intelligence 4) Energy Storage 5) Robotics

• These 5 platforms are supported by substreams of - Cryptocurrencies, Smart Contracts, Digital Wallets, Multiomic Technologies, Precision Therapy, Programmable Biology, Neural Net, Next Gen Cloud, Intelligent Devices, Adaptive Robotics, 3D Printing, Reusable Rockets, Autonomous mobility, Advanced Battery Technology

Resources

• A concise and comprehensive technical paper on the LLM landscape - architectural innovations, better training strategies, context length improvements, fine-tuning, multi-modal LLMs, robotics, datasets, benchmarking, efficiency, and more. To give a sense of the extent of the research, the paper references 485 papers from LLM space! (Humza Naveed et al, University of Technology & Engineering, Lahore & Multiple universities)