private:unsupervised

This shows you the differences between two versions of the page.

Both sides previous revision Previous revision | |||

private:unsupervised [2020/03/05 08:30] admin [Why is dot product so dominant?] |
private:unsupervised [2020/03/05 09:50] (current) admin [Orthonormal weights] |
||
---|---|---|---|

Line 72: | Line 72: | ||

* exact methods - decompose a (n,n) matrix into (n-1, n-1) and ensure each step maintains orthonormal - has to be n steps to backprop through which can't be good. | * exact methods - decompose a (n,n) matrix into (n-1, n-1) and ensure each step maintains orthonormal - has to be n steps to backprop through which can't be good. | ||

* approximate methods - enforce rows or columns to be unit vector and then minimise Tr(W W^T - I) using (say) LMS | * approximate methods - enforce rows or columns to be unit vector and then minimise Tr(W W^T - I) using (say) LMS | ||

+ | * near exact - use exp(W) and approximate the exp - http://proceedings.mlr.press/v97/lezcano-casado19a/lezcano-casado19a.pdf | ||

==== Embedding daat onto unit sphere === | ==== Embedding daat onto unit sphere === |

private/unsupervised.txt ยท Last modified: 2020/03/05 09:50 by admin