Neural Networks

Assignment 7: Vector Symbolic Architectures

- Enhance our understanding of VSA by implementing the “Dollar of Mexico” analogy example from Kanerva (2008).
- Understand the use of permutation to implement sequencing.

>>> from vsa import randvec >>> randvec(10) array([-1, -1, 1, 1, -1, -1, -1, -1, -1, -1]) >>> randvec(10) array([ 1, 1, -1, -1, -1, 1, 1, -1, -1, -1]) >>> randvec(10) array([-1, -1, -1, -1, 1, -1, -1, 1, 1, 1])Next, you can copy/paste your vector

>>> from vsa import randvec, cosine >>> a = randvec(1000) >>> b = randvec(1000) >>> c = randvec(1000) >>> d = randvec(1000) >>> x = a*b + c*d # Associate A with B and C with D >>> y = c * x # Probe the association vector with C; answer should be D >>> cosine(y, a) -0.03508232077228117 >>> cosine(y, b) -0.052623481158421748 >>> cosine(y, c) -0.029235267310234306 >>> cosine(y, d) 0.68410525505948272As you can see, the query worked as expected: when we query the association vector

- a constructor accepting the vector size
`n` - a method to create a random vector with a name you specify as a string
- a method to return the name of the “winner” (most likely) vector in a query

>>> from vsa import VSA >>> vsa = VSA(1000) >>> a = vsa.randvec('A') >>> b = vsa.randvec('B') >>> c = vsa.randvec('C') >>> d = vsa.randvec('D') >>> x = a*b + c*d >>> y = c*x >>> vsa.winner(y) 'D'All this is pretty easy to do if your

>>> vsa = VSA(10000) # Ten thousand elements, as per Kanerva (2008) >>> nam = vsa.randvec('name') >>> usa = vsa.randvec('USA') >>> cap = vsa.randvec('capital') >>> wdc = vsa.randvec('Washington, DC') >>> mon = vsa.randvec('money') >>> dol = vsa.randvec('dollar') >>> mex = vsa.randvec('Mexico') >>> mxc = vsa.randvec('Mexico City') >>> pes = vsa.randvec('peso') >>> ustates = nam*usa + cap*wdc + mon*dol >>> mexico = nam*mex + cap*mxc + mon*pes >>> fum = ustates * mexico >>> query = dol*fum >>> vsa.winner(query) 'peso'

>>> vsa = VSA(10) >>> x = vsa.randvec('X') >>> y = vsa.permute(x) >>> x array([ 1, 1, 1, 1, 1, -1, -1, 1, 1, 1]) >>> y array([-1, 1, 1, 1, 1, 1, 1, 1, 1, -1]) >>> vsa.perminv(y) array([ 1, 1, 1, 1, 1, -1, -1, 1, 1, 1])

- Start with a result vector of zeros.
- For each symbol in the sequence, last to first:
- Add the vector for this symbol to the current result vector
- Permute the result vector
- Return the result vector

Why does our loop go backwards over the sequence (step 2)? If you think about it, this encoding is a bit like a stack: the last symbol added will be at the “top” of the result vector. So to ensure that the first symbol in the sequence is the last one added, we loop over the sequence in reverse order.

To test your `seqencode` method, you'll need to add a `seqdecode` method. This method
should take a vector returned by `seqencode` and return the list of symbols encoded by that
vector. Another look at Slide 17 of the lecture notes gives us the algorithm for this method:

- Start with an empty result list.
- Until the entire sequence has been decoded:
- Replace the vector by its inverse permutation
- Get the winning symbol for the vector and append this symbol to the result list
- Return the result list

Fortunately, the answer to this puzzle lies in the methods you have already written. Rather than giving
you the answer, I'll provide a hint: look again at the code inside the `winner` method you invoke in step 4
of the algorithm. If you are still puzzling over this after thinking about it for a while, ask me, and
we'll work it out together.