Business: $100B Infrastructure fund from Microsoft & Black Rock, App building with Together AI's Llama coder, Apple's beta release of Apple Intelligence, Google Vs US trial - evidence on monopolistic pricing, Full break down of USA Federal budget which is running at $6.2Trillion per annum

Technology: Creating Synthetic data set using Llama 3.1 405B model & NVIDIA Nematron model, Prompt optimisation framework called DSPy, Choosing between LLM Agent frameworks, Finetuning Llama 3 8B model on Together AI with Math Instruct dataset

Resources: AI research paper video log from Arize.AI, UC Berkeley's graduate NLP course (CS288) assignment set, Together AI research blog on open source LLM engineering & research

AI in Businesses

• Infrastructure-oriented (for new and existing data centers) fund from a consortium of companies headed by Microsoft and BlackRock with an initial $ 30 billion capital which will grow to $ 100 billion in future.  The funds are still pouring in billions every week and month despite the scepticism around the real returns on AI investments(Jordan Novet, CNBC)

• Generate an entire app with Together AI's LlamaCoder - this article is in anyways an advertisement of what is possible with open-source Llam 3.1 models (405B in this case). It's getting 'No Code' when it comes to small and fun consumer applications already, the distance to a more serious back-office enterprise application should not be too far in future. This is a constant theme - the threat to subscription-based software from AI-generated software. There are 200,000 applications generated since launch (Meta's Blog)

• Keenly watching this consumer play of AI through Apple's massive device proliferation and all the possibilities it comes with - Apple released Apple Intelligence on public Beta for iPhone 15 Pro and onwards (Jay Peters, The Verge)

• The U.S. vs Google antitrust trial has brought to light that Google retained an undisclosed take rate of 32% from publishers for selling their ad space. Additionally, the buy and sell sides of Google's advertising ecosystem operated on separate platforms, leading to a lack of correlation between supply and demand. Internally, Google considered implementing an open auction system or capping its margins. However, it chose to retain the rates, likely because it did not face significant competition in its domain, which strengthens the argument in the antitrust trial. The original email exchange can be found on the DOJ website and referenced by outlets like The Verge. (The Verge, Justice.gov)

• Steve Balmer published a very useful fact disclosure on US Federal budget and its debt position. The numbers look scary. Collationist provides a short commentary - 38% budget deficit every year and 15% of the Federal revenue going to interest payments - doesn't seem to be sustainable! The numbers itself published in this blog (Dennis Kuriakose, Collationist.com)

Technology updates from AI

• Creating synthetic dataset using Llama 3.1 to fine-tune your LLM - the grand idea is creating a 500 prompt / response pair for a 'Git command' prompt problem which can be later used to fine-tune a smaller LLM for personal or enterprise use. Llam 3.1 405B model is hosted in NVIDIA NIM containers and exposed as APIs. Further NVIDIA Nematron is used as a reward model to filter out low-quality pairs (Hesam Sheikh, Towards Data Science - Medium)

• Stanford researchers have developed and refined a workflow and prompt optimization framework called DSPy. DSPy simplifies prompt writing and model fine-tuning by introducing a modular structure, creating a layer of abstraction between large language models (LLMs) and end-user applications. This marks a significant advancement over retrieval-augmented generation (RAG) frameworks like Langchain, which require developers to craft manually and continually refine prompts with each new model release.  A blog summarising the conversation technology highlights from Collationist. (MLOps.community YouTube channel)

• Choosing Between LLM Agent Frameworks discussed the trade-offs between building bespoke code-based agents and the major agent frameworks. This article gives a very high-level option analysis and a set of architecture templates to get you started. (Aparna Dhinakaran, Medium.com/Arize.AI)

• Together AI allow customers to run inference services on open source models. They also allow customers to fine-tune open-source models and then run the inference services on the resulting model. An example is fine-tuning the Llama 3 8B base model with the Math Instruct data set and then deriving comparable results to that of Chat GPT -4 and Llam 3 70B models at a fraction of the prices (~$100 to train on a 207K data set). (Hassan El Mghari, Together AI)

Resources

• Arize AI is an AI observability and LLM evaluation platform. The team at Arize.AI regularly discusses AI research papers through video blogs, often featuring insights from industry experts to explore the latest advancements in AI.

• Interactive assignments for teaching structured neural NLP were created for UC Berkeley's graduate NLP course (CS288). These Spring 2020 assignments continue to offer a solid foundation for building essential NLP skills. While only the first three notebooks are publicly available, they cover key concepts ranging from part-of-speech tagging classifiers to transformers, providing a strong starting point for learners.

• Together AI research blog - their research team contributes cutting-edge models, datasets, and optimizations to the open-source community