Beyond Moore's Law: How Light is Revolutionizing Computing

The End of an Era: Moore's Law and Its Limits

For decades, we've relied on Moore's Law, the principle that the number of transistors on a microchip doubles roughly every two years. This has fueled the digital revolution, enabling everything from smartphones to the internet. However, we're now reaching the fundamental limits of silicon-based electronics. Transistors are approaching the size of individual atoms, and further miniaturization is becoming impossible due to quantum and thermodynamic uncertainties. Simply put, we can't shrink them any further.

This has led to a shift in how we approach improving computer performance. Instead of making transistors smaller, the focus is now on making computers bigger and more interconnected. Simultaneously, our ambitions for artificial intelligence are skyrocketing. The last seven years have seen incredible advancements in AI, with compute power increasing dramatically. However, networking is now becoming the bottleneck. Achieving Artificial General Intelligence (AGI) will require vast amounts of computational power and data, potentially far beyond what current technology can handle.

So, is this the end of computational progress as we know it? Or is there a path beyond the atom, towards the speed of light?

The Rise of Photonics: Computing with Light

Many believe the answer lies in photonics, the science and technology of generating, detecting, and controlling photons, or light. While photonics has been a popular subject in computational literature, it has never seen mainstream commercial usage until now. We've reached a point where we can build photonics at a scale previously unimaginable.

Light has been revolutionizing how we send information since the 1800s, with the advent of flashing lights and mirrors for signaling. The real breakthrough came in the 1960s with fiber optic cables, thin strands of glass that trap light and use it to transmit data. This technology now underpins most of the internet, with vast networks of undersea cables connecting us globally. While we've mastered sending information using light, computing with it has proven much more challenging.

Scientists have dreamed of optical computers since the 1980s because light is much faster and more efficient than electricity. However, controlling light for computation is incredibly tricky. Light tends to spread out and scatter, and getting it to make sharp turns or stay in tiny spaces goes against its nature. It's only now that science and companies like Lightmatter are making photonics a viable technology.

Unlike traditional electronics that use electrical signals through wires, photonics uses optical fibers or waveguides on a chip. This allows data to be transmitted faster over longer distances with less energy loss. Lightmatter is at the forefront of this revolution, integrating photonic components directly into computer chips, enabling a new method of chip communication that could pave the way for AGI.

Lightmatter: Pioneering the Future of Computing

To better understand this technology and the limitations of current computational methods, we visited Lightmatter's Palo Alto lab and spoke with CEO and co-founder Nick Harris.

"We are enabling these AGI supercomputers that are being built toward realizing these machines that are able to think at the same caliber as the best humans on the planet," says Nick Harris.

The Challenge of AGI

AGI, or Artificial General Intelligence, is an AI that can understand, learn, and perform any intellectual task at a human level or better. The recent AI boom has made this goal seem more tangible, but achieving it requires a massive leap in computational power.

The End of Dennard Scaling

For decades, we also benefited from Dennard scaling, which stated that as transistors got smaller, they would use less power and run faster. This is why your phone today is more powerful than a room-sized computer from the 1970s. However, around 2006, this free lunch ended. We hit fundamental limits of physics, and making chips better became much harder and more expensive. This is why companies like Lightmatter are so exciting – they're trying to reinvent computing itself.

Light's Potential

Nick Harris's journey into computing began with gaming and programming. He eventually worked at a semiconductor company, studying transistors. He realized that there were fundamental challenges in continuing to improve computers using traditional methods. This led him to explore the potential of light. He went to MIT to study photonics using silicon, the same material used to build computer chips. He also studied quantum information theory and quantum computing.

"What I learned along the way was intricately how light works and what the future might look like with light compared to electrons," says Harris.

Light is ideal for communication and transmission of information. It enables faster and more reliable interconnects over long distances because it travels at the speed of light. It also generates less heat and consumes less energy, making it sustainable for large-scale systems like AI supercomputers.

Lightmatter's Products: Passage and Envise

Lightmatter is the first company to commercialize photonics technology. They have two main products: Passage and Envise.

Passage is a 200mm by 200mm chip built in 2019. It consists of an array of 48 maximum-size chips stitched together with optical waveguides. These waveguides allow light to be manipulated and encoded with information, enabling GPUs and switches to build data center-scale supercomputers. Optical fibers, similar to those used for internet connections, are attached to the side of the photonic chip. By programming the chip, the connections between the 48 tiles can be configured. This allows customer GPUs, switches, or CPUs to communicate using light.

"By using silicon photonics and using light, we would actually be able to drive the energy efficiency, drive the speed, and continue the roadmap for computing," explains Harris.

The next-generation Passage chip can communicate at over 100 terabits per second. It has 256 attached fibers, the largest optical fiber count in the world, and the highest bandwidth chip in the world. It acts as a superhighway for data passing between different chips, enabling unprecedented bandwidth and ultra-low latency communication. This addresses the "shoreline problem" where there isn't enough space to plug electrical wires into the chip. Photonics sidesteps this issue by using optical waveguides within the chip itself, allowing data to be transmitted through the entire chip, massively increasing potential bandwidth.

Envise is Lightmatter's all-in-one photonic computing chip. It combines traditional electronics with photonics to perform computations using light. This is a completely different way of thinking about computation. Data flows through the computer at the speed of light, using different colors to process multiple things simultaneously. Traditional computers are limited by the speed of electrical signals, but light moves through these chips without delay. Lightmatter also stacks what they call "virtual processors" on top of one another, creating multiple computers in the same physical space. All of this is achieved while using far less power than regular chips.

The Bottleneck: Interconnects and Networking

As overall computing performance increases, the bottleneck is no longer the chips themselves but the networking or interconnects between them. Networking is the communication and data exchange between multiple processors or chips within a computing system. It's critical for efficient performance in modern multi-chip systems.

"The last seven years has been an incredible time for AI and a lot of what I saw is the unbelievable increase in the pace of computing... but a lot of those advances are starting to hit certain walls and most of that has to do with the connectivity," says Harris.

Transistors are now so small that they're approaching the size of an electron, meaning we can't shrink them anymore. The world has decided that the first path towards making computers better is making them bigger. However, the challenge is how to interconnect them at scale. The limitations of transistors are troubling because progress in almost every field is tied to progress in computing. If progress slows down, it will impact our ability to simulate the world, perform calculations, and connect in real-time.

The Shoreline Problem

Steve Clinger, VP of product at Lightmatter, explains that the size and complexity of AI models have increased dramatically. To train these large language models, you need a huge number of compute units that can communicate at very high speed with very low latency. In traditional designs, the points where data escapes are limited by the electrical shoreline of the chip. You run out of space to put the high-speed signals that are sending all this data. Lightmatter's technology allows bandwidth to escape anywhere on the chip, dramatically scaling the amount of bandwidth that can be simultaneously communicated outside of these chips.

"What we're doing is we're enabling these supercomputers to act as a single massive chip," says Clinger.

Lightmatter is replacing the traditional silicon substrate with a new one that enables chips to send data using light. This allows them to slipstream next-generation technology into the existing ecosystem.

Lightmatter's Clean Room: Building the Future

Lightmatter has built its own mini-fabrication facilities and uses advanced microscopy to develop and inspect their chips. This allows them to rapidly prototype and iterate on their designs. R Tesh Jane, senior VP at Lightmatter, gave us a tour of their clean room facility.

"This is an end-to-end capability where we're going from a silicon that's showing up on a wafer all the way out to a packaged unit that has fibers attached that can be deployed into a system," explains Jane.

The clean room includes the fiber attach process, where fibers are aligned and attached to the chip. The waveguides, which are etched into the silicon, guide the light into the chip. Each fiber carries 16 colors of light, all around a wavelength of 1300 nm. The light is coupled into the waveguide, which is a few hundred nanometers wide and about 100 nanometers tall. Inside this tiny device, 16 colors of light travel together.

The Physics of Light

Light is interesting because it relies on a fundamentally different set of physics than electronics. Electronics are governed by properties like resistance, inductance, and capacitance, which describe power dissipation and switching speed. Light, on the other hand, doesn't have these properties. It operates in the hundreds of terahertz regime, while electronics operate in the gigahertz regime. Light also enables parallel communication in a way that electrical signals cannot.

Silicon photonics has been developed over the past 20 years, but the challenge was that there were no foundries or commercial partners to scale the technology. Additionally, electrical wires were still making sufficient progress, so there wasn't a strong push to explore the potential of light. Lightmatter has built photonics at a scale that no one has ever seen before, with millions of optical components on a chip.

"I love chips. Every time I get to see one of the chips that we're building, it blows my mind," says Harris.

Conclusion: A New Era of Computing

Developing this kind of technology isn't easy. It requires significant investment, cutting-edge research, and the perseverance to overcome technical challenges. However, the potential rewards are immense. The death of Moore's Law doesn't mark the end of technological or economic progress. Instead, it marks the beginning of a new age of computing, limited only by the speed of light. Lightmatter is at the forefront of this revolution, paving the way for a future where light is the driving force behind computation and AI.