Aye that was a stoater as well
"I recently designed a rug collection " Did ye, aye?
Sorry @happybeing but this is a target-rich environment
Have you been hanging upside down too long?
2 Likes
Didn’t you read my content warning?
1 Like
Don’t know you know @Southside I could definitely see you doing some late night UV hula hooping 
1 Like
=====WARNING SEXIST AGEIST CONTENT==============
What size is Anneka Rice’s arse now?
Used to be “meaty but munchable” but that was 30+ years ago. I used to enjoy her getting out of the helicopter…
1 Like
Only if you promise to help me take down the pictures and put away the ornaments first.
You bring the hoops, I’ll do the UV and @aatonnomicc can get the beers. Its his round anyway.
Until then, these are the only hoops I will be seen in
1 Like
FTX: Alameda had unlimited access to FTX customers’ funds 
3 Likes
Another FTX/Alameda domino falls: Genesis
3 Likes
1 Like
We build a Generatively Pretrained Transformer (GPT), following the paper “Attention is All You Need” and OpenAI’s GPT-2 / GPT-3. We talk about connections to ChatGPT, which has taken the world by storm. We watch GitHub Copilot, itself a GPT, help us write a GPT (meta :D!) . I recommend people watch the earlier makemore videos to get comfortable with the autoregressive language modeling framework and basics of tensors and PyTorch nn, which we take for granted in this video.
Chapters: 00:00:00 intro: ChatGPT, Transformers, nanoGPT, Shakespeare baseline language modeling, code setup
00:07:52 reading and exploring the data
00:09:28 tokenization, train/val split
00:14:27 data loader: batches of chunks of data
00:22:11 simplest baseline: bigram language model, loss, generation
00:34:53 training the bigram model
00:38:00 port our code to a script Building the “self-attention”
00:42:13 version 1: averaging past context with for loops, the weakest form of aggregation
00:47:11 the trick in self-attention: matrix multiply as weighted aggregation
00:51:54 version 2: using matrix multiply
00:54:42 version 3: adding softmax
00:58:26 minor code cleanup 01:00:18 positional encoding
01:02:00 THE CRUX OF THE VIDEO: version 4: self-attention
01:11:38 note 1: attention as communication
01:12:46 note 2: attention has no notion of space, operates over sets
01:13:40 note 3: there is no communication across batch dimension
01:14:14 note 4: encoder blocks vs. decoder blocks
01:15:39 note 5: attention vs. self-attention vs. cross-attention
01:16:56 note 6: “scaled” self-attention. why divide by sqrt(head_size) Building the Transformer
01:19:11 inserting a single self-attention block to our network
01:21:59 multi-headed self-attention
01:24:25 feedforward layers of transformer block
01:26:48 residual connections
01:32:51 layernorm (and its relationship to our previous batchnorm)
01:37:49 scaling up the model! creating a few variables. adding dropout Notes on Transformer
01:42:39 encoder vs. decoder vs. both (?) Transformers
01:46:22 super quick walkthrough of nanoGPT, batched multi-headed self-attention
01:48:53 back to ChatGPT, GPT-3, pretraining vs. finetuning, RLHF
01:54:32 conclusions
Corrections:
00:57:00 Oops “tokens from the future cannot communicate”, not “past”. Sorry! 
2 Likes
I don’t know the limits of what ChatGPT is but an AI that could identify fallacies would be something special. Imagine that put against politics and how it could feedback to encourage a better quality of debate.
5 Likes
It will need to do more than point then out but present irrefutable evidence and reasoning. Otherwise you’re back to the question of trust and the risk of error, accidental or deliberate bias, manipulation and control.
It’s the reason we have science and that has been seriously undermined by the things I just listed being weaponised on online platforms.
4 Likes
Privacy. Security. Freedom
3 Likes
It’s about time!
The NRC accepted NuScale’s SMR design certification application back in March 2018 and issued its final technical review in August 2020. The NRC Commission later voted to certify the design on July 29, 2022—making it the first SMR approved by the NRC for use in the United States.
Bureaucracy in action … errr, inaction. Talk about feet dragging.
Oh, but guess what, still more bureaucracy:
NuScale is currently seeking an uprate to enable each module to generate up to 77 megawatts. The NRC is expected to review their application this year.
“expected to review” … this year … lmao
The first module is expected to be operational by 2029 with full plant operation the following year.
Unreal … but just inline with UN and WEF 2030 agenda.
1 Like