how_to_start_up_a_startup
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision | Next revision Both sides next revision | ||
|
how_to_start_up_a_startup [2020/04/27 08:48] admin [Example: Neuracore.ai] |
how_to_start_up_a_startup [2020/04/27 08:52] admin [Example: Neuracore.ai] |
||
|---|---|---|---|
| Line 146: | Line 146: | ||
| In order to make an impact any solution must be complete, that is almost invisible to the user. It needs to improve on the three major operations: | In order to make an impact any solution must be complete, that is almost invisible to the user. It needs to improve on the three major operations: | ||
| - | * fwd: The inference, or forward pass of a DNN model | + | * forward: The inference, or forward pass of a DNN model |
| - | * bwd: The backward error propagation pass of stochastic gradient descent | + | * backward: The backward error propagation pass of stochastic gradient descent which accumulates gradients over a batch |
| - | * acc: The combination of fwd and bwd results to get and error signal which is accumulated over a batch | + | * update: The work needed to scale the batch gradient into a weight update (may need complex CPU like operations) |
| - | The final pass, model update, can be formulated as computationally lower cost (e.g. updating only whenever there is a significant change) and also is not standardised in approach (ref: [[http://ruder.io/optimizing-gradient-descent|S. Ruder]]). There are also other operations (e.g. softmax and batch normalisation) that are best suited to general a purpose processor. | + | |
| == Guiding Principles == | == Guiding Principles == | ||