engineeredbygaurang.com

← All writing

First Principles thinking

Learnings from Jensen Huang

Jun 30, 2025 4 min read

Power of LLMs helps to learn from great leaders and their history.

Below is a snippet from ChatGPT conversation on my quest to learn first principles thinking shared by NVIDIA CEO Jensen Huang:

1. Break It Down to the Basics

Instead of copying how others solve a problem, strip it down to:

  • What are the laws of nature, constraints, or foundational facts?

  • What do we know for sure is true?

Example: When NVIDIA pivoted into AI and data centers, Huang didn’t just follow trends. He looked at:

  • How computing works fundamentally

  • The raw power of parallel processing (a GPU's strength)

  • The future needs of AI workloads

That led to the insight: GPUs could do much more than graphics.

2. Rebuild with Fresh Eyes

Once you're at the core, rebuild the solution from the ground up. Think:

“If we had to start from scratch, knowing what we know now, how would we do this?”

Huang's Quote:

“You can learn how something can be done and then go back to first principles and ask yourself, ‘Given the conditions today… how would I reinvent this whole thing?’”

3. Avoid the Trap of “This Is How It’s Always Done”

First-principles thinkers don’t assume best practices are always right. Jensen encourages teams to challenge assumptions, even if it means discarding years of conventional wisdom.

Why This Matters at NVIDIA

This mindset let NVIDIA:

  • Move beyond gaming into AI, robotics, and autonomous vehicles

  • Create CUDA (a programming language for GPUs) when no one else was thinking that way

  • Build chips like the H100, tailored for deep learning from scratch

Pivotal moment in NVIDIA’s journey where Jensen Huang’s leadership and first-principles mindset shaped the company’s future:

The Moment: Launching CUDA (2006)

Context:

In the early 2000s, GPUs were primarily for rendering graphics. Scientists and engineers started realizing GPUs might help with scientific computing—but programming them was nearly impossible without deep knowledge of graphics APIs.

The Bold Move: Inventing CUDA

Jensen asked:

“What if we built an entire programming platform that lets developers use GPUs like general-purpose supercomputers?”

This had never been done. No other company was thinking of turning graphics chips into compute engines.

First-Principles Thinking in Action:

Instead of assuming:

“GPUs = graphics, and should be controlled through graphics APIs like OpenGL,”

They asked:

“What if the GPU is just a highly parallel processor—and we gave people tools to program it directly?”

That idea became CUDA (Compute Unified Device Architecture)—launched in 2006.

Why This Was Risky:

  • It took years of R&D.

  • Developers didn’t know how to use it.

  • Wall Street didn’t get it—investors questioned why NVIDIA was pouring resources into something so “out of scope.”

But Jensen trusted the vision and invested through the skepticism.

Long-Term Payoff:

  • CUDA became the backbone for AI research.

  • When deep learning exploded in the 2010s (thanks to ImageNet breakthroughs), NVIDIA was the only company with the right hardware and software stack ready.

  • CUDA + GPUs = dominant AI infrastructure.

Leadership Takeaway:

Jensen leads like a founder-engineer:

  • He doesn’t chase fads—he prepares for inevitabilities.

  • He empowers engineers to take moonshot risks and backs them with time and patience.

  • He builds tools for developers, not just products—creating ecosystems, not just hardware.

Steve Jobs in 1983 predicting LLMs: https://tinyurl.com/429pradn

Chat GPT conversation reference: https://chatgpt.com/share/67f57382-f474-800b-b6af-7152e4b32673