Sebastian Raschka, PhD LinkedIn Posts

One of the big bottlenecks with LLMs and Vision Transformers is GPU memory on consumer devices. I just wrote about my favorite techniques for

Just wrote a new article on "Improving LoRA: Implementing Weight-Decomposed Low-Rank Adaptation (DoRA) from Scratch": https://lnkd.in/gJk48kms

I am super excited about DoRA, and based

Just wrote a new article on "Improving LoRA: Implementing Weight-Decomposed Low-Rank Adaptation (DoRA) from Scratch": https://lnkd.in/gJk48kms

I am super excited about DoRA, and based

While everyone is talking about Sora, there's a potential successor to LoRA (low-rank adaptation) called DoRA. Here's a closer look at the "DoRA:

Another quality, well researched article by Sebastian Raschka, PhD.
Saved me a lot of time with finetuning various Huggingface LLMs for text classification

Written & done! Just finished Chapter 4 on implementing an LLM architecture, which marks the marks the 50% point of the book.
(I

Can "small" finetuned LLMs with less than 2B parameters outperform larger openly available LLMs (Mixtral, Llama 2 Chat) and proprietary LLMs (ChatGPT)? Here's

I've primarily focused on finetuning LLMs over the past few months and have recently started to explore model merging. As part of that

In this episode of Leading With Data, we interact with Sebastian Raschka, PhD, AI Staff Educator at Lightning AI.

Sebastian Raschka, PhD is

Looking at open source and research in 2024 so far, it seems we are moving towards making LLMs better (and smaller) without necessarily

Proxy-tuning is a way to adapt LLMs without changing the model's weights. This is especially attractive if a given LLM is too resource-intensive

There's a new promising method for finetuning LLMs without modifying their weights called proxy-tuning.
How does it work? It's a simple decoding-time method where you

I just saw that my Ahead of AI magazine crossed the 20k subscriber mark!

I am incredibly grateful for all the support. Knowing

Feeling a tad bit bored by decoder-only transformers like GPT and LLaMA? Let's delve back into the the world encoder-style BERT models for

A new research paper just came out that proposes an alternative to reinforcement learning with human feedback (RLHF), which is used to finetune

"LIMA: Less Is More for Alignment" might be a game-changer for researchers and tinkerers who want to develop capable LLMs.

In this paper,

It's been many months in the making, and I am excited to share that the print version of Machine Learning Q and AI

When doing machine learning and AI research (or writing books), making the code reproducible is usually desirable. Often, that's easier said than done!

A good start to the new week: The highly requested Chapter 5 of my "Build an LLM from Scratch" book is finally available,

We just added CodeGemma 7B support to LitGPT by a kind contribution from Andrei Aksionau.
Playing around with it for a bit, I think