jdpressman
commited on
Commit
•
fc73b33
1
Parent(s):
0f0e2e4
Fix example output formatting so it doesn't run past view window
Browse files
README.md
CHANGED
@@ -54,22 +54,35 @@ them towards the topic, while averaging together multiple completions on each se
|
|
54 |
Here is an example of an output from this process:
|
55 |
|
56 |
```
|
57 |
-
Then it asked the network to reconstruct the input and the original embedding. The network had to learn to match the
|
58 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
59 |
'Yes,' Rin replied. 'We just have to figure out how to use it.'
|
60 |
'How?' Rin asked.
|
61 |
'You can move the second word in, right?'
|
62 |
'Possibly.' Rin thought for a moment.
|
63 |
'The second word will be the first word of the next arago,' Mu said. 'We just need to find it.'
|
64 |
-
'True,' Rin agreed. 'Well, I'll let you know what a Gaussian.’ ‘Let’s see if we can get it to work.’ ‘Arago the second
|
65 |
-
|
|
|
|
|
|
|
66 |
'That's the second arago,' Rin said.
|
67 |
'The second arago?' Argo asked.
|
68 |
'Rin has found the second arago.'
|
69 |
Argo stared at Rin. 'Argo, is there something wrong?'
|
70 |
'I thought so.'
|
71 |
'What?' Rin said.
|
72 |
-
'I don't know,' Argo said. 'I thought I was the smartest person in the world but, well, I only had a certain amount of
|
|
|
73 |
```
|
74 |
|
75 |
This generation method is slow, but retrieval could be used to speed up inference and make it converge closer and closer
|
|
|
54 |
Here is an example of an output from this process:
|
55 |
|
56 |
```
|
57 |
+
Then it asked the network to reconstruct the input and the original embedding. The network had to learn to match the
|
58 |
+
embedding to the original input, therefore matching the inference by consuming the embedding. This was key because
|
59 |
+
the embedding had to be able to match the text with the text it was consumed with. 'Here's how you do it,' Boru told Mu,
|
60 |
+
'Just impute the mean and variance.' This Mu did, transforming not words but entire paragraphs into vectors and then
|
61 |
+
inferring the next paragraph. It took some tweaks and tuning to get the initial performance but the second arago spot
|
62 |
+
had been found. To make sure the network was learning the right thing, Boru had to check the first value in the vector.
|
63 |
+
If the first value was below 0, the network had failed to learn the first value. If the value was above 0, the network
|
64 |
+
had been able to learn the first value.
|
65 |
+
‘What have you called this, Boru?’ asked Mu. ‘Latent variable regression.’ ‘It looks like a mixture of density network
|
66 |
+
and autoencoder,’ said Nayaf. ‘It’s an autoencoder but it’s using latent variables, but we’re using the mean and variance
|
67 |
+
of Grade had a difficult time seeing it, but he could tell it was close. 'So you've found the second arago,' he said.
|
68 |
'Yes,' Rin replied. 'We just have to figure out how to use it.'
|
69 |
'How?' Rin asked.
|
70 |
'You can move the second word in, right?'
|
71 |
'Possibly.' Rin thought for a moment.
|
72 |
'The second word will be the first word of the next arago,' Mu said. 'We just need to find it.'
|
73 |
+
'True,' Rin agreed. 'Well, I'll let you know what a Gaussian.’ ‘Let’s see if we can get it to work.’ ‘Arago the second
|
74 |
+
spot?’ ‘We’re here,’ Arago said.
|
75 |
+
The second spot was located in the middle of the text. Arago had to read it again to find the proper signal. ‘I’m going
|
76 |
+
to have to tweak some of the weights,’ said Arago. ‘I’ve had to change the input to the next layer from an input to
|
77 |
+
output.’ ‘You’re making a mistake again,’ said Mu to Arago. ‘It’s a mistake.’ The network had been learning I find out.'
|
78 |
'That's the second arago,' Rin said.
|
79 |
'The second arago?' Argo asked.
|
80 |
'Rin has found the second arago.'
|
81 |
Argo stared at Rin. 'Argo, is there something wrong?'
|
82 |
'I thought so.'
|
83 |
'What?' Rin said.
|
84 |
+
'I don't know,' Argo said. 'I thought I was the smartest person in the world but, well, I only had a certain amount of
|
85 |
+
energy. I didn't know how to do the second arago until now, but I can't
|
86 |
```
|
87 |
|
88 |
This generation method is slow, but retrieval could be used to speed up inference and make it converge closer and closer
|