I think most people underestimate how big of a deal it’s going to be when this tech is pervasive in things like search engines or digital assistants. There are many times when I can’t figure out the right combination of words to put into a search engine to find the results. ChatGPT is already my go to when I want to figure out a movie or song from some random combination of foggy memories. Imagine after 10 more years of cpu/gpu innovations, and chat applications that have actually been designed for information retrieval, how much that is going to transform how we interact with data and information.
Full disclosure, I didn’t watch the video. I just can’t imagine that that headline isn’t going to look silly in 30 years.
Imagine after 10 more years of cpu/gpu innovations, and chat applications that have actually been designed for information retrieval, how much that is going to transform how we interact with data and information.
LLMs are going to change how we interact with data and information, but not the way you think. The AI-generated spam will ruin the whole concept of internet search completely. Only information that we can trust is going to be human-curated.
There are diminishing returns in semiconductor photolitho. Moore scaling is long over, absolute real estate see WSI with Cerebras, DC costs and power envelope are all sending a clear message. Quantization is there, so you can go from digital multipliers to analog and go spiking networks, but transformers and Co have little power there.
Also, the kind of economy that can carry Gen AI as business model is not a given, long term.
Neuromorphic hardware is going to jump many orders of magnitude over classic hardware. When we get a RAM that can execute multiple layers in parallel at once, per clock tick, we’ll see whole AI ecosystems cooperating to get a solution in a fraction of the time a single modern NN would take.
Yes orders of magnitude, but not too many of them. The real estate of a 300 mm wafer is limited, the structure shrink is saturating and you can’t get too many layers. You still need a packet switched network on the wafer even if the rest is mostly analog. Perhaps spintronics can limit the power requirements too.
dick_stitches@lemm.ee 1 year ago
I think most people underestimate how big of a deal it’s going to be when this tech is pervasive in things like search engines or digital assistants. There are many times when I can’t figure out the right combination of words to put into a search engine to find the results. ChatGPT is already my go to when I want to figure out a movie or song from some random combination of foggy memories. Imagine after 10 more years of cpu/gpu innovations, and chat applications that have actually been designed for information retrieval, how much that is going to transform how we interact with data and information.
Full disclosure, I didn’t watch the video. I just can’t imagine that that headline isn’t going to look silly in 30 years.
FreeFacts@sopuli.xyz 1 year ago
LLMs are going to change how we interact with data and information, but not the way you think. The AI-generated spam will ruin the whole concept of internet search completely. Only information that we can trust is going to be human-curated.
jarfil@beehaw.org 1 year ago
You will need an LLM to tell that apart, so… 🤷
eleitl@lemmy.ml 1 year ago
There are diminishing returns in semiconductor photolitho. Moore scaling is long over, absolute real estate see WSI with Cerebras, DC costs and power envelope are all sending a clear message. Quantization is there, so you can go from digital multipliers to analog and go spiking networks, but transformers and Co have little power there.
Also, the kind of economy that can carry Gen AI as business model is not a given, long term.
jarfil@beehaw.org 1 year ago
Neuromorphic hardware is going to jump many orders of magnitude over classic hardware. When we get a RAM that can execute multiple layers in parallel at once, per clock tick, we’ll see whole AI ecosystems cooperating to get a solution in a fraction of the time a single modern NN would take.
eleitl@lemmy.ml 1 year ago
Yes orders of magnitude, but not too many of them. The real estate of a 300 mm wafer is limited, the structure shrink is saturating and you can’t get too many layers. You still need a packet switched network on the wafer even if the rest is mostly analog. Perhaps spintronics can limit the power requirements too.