Development Log for Neural Network PrototypeThe idea, at the core:
Port a limited RNN/LSTM Neural Network model from Python with a reduced training set and dimension size for layers to demonstrate that a fully functional (even if limited) Neural Net can work in Pine.
Limited model + having the python code on hand = Able to test and verify components in Pine at every step, in theory
The model/script I'm attempting to implement a limited subset of is detailed here:
iamtrask.github.io
A dataset in binary is required, but binary does not exist in pinescript, thus:
To do this, decimal to binary and binary to decimal functions are required. This didn't exist previously - I've written a script to accomplish just that:
Originally, this was going to have a input_dim of 2, hidden_dim of 16, but I've changed the hidden_dim to 8 (binary dimensions from 8 to 5) to reduce the dataset range to max 32 while I figure out to implement working pseudo-arrays and state updates. I've looked at RicardoSantos's scripts for Markov and Pseudoarrays, and will be using them as a reference going forward.
I've verified the output of the Sigmoid function and 1st derivative of the Sigmoid function in Python for values of (-1,0,0.5,1 ). I've yet to publish the Sigmoid script pending approval from TV moderators about including python code that is commented out at the bottom to verify the results of that script.
What I'm trying to do here with training dataset generation was unsuccessful, for multiple reasons:
Lack of formal array constructs in pine
Psuedorandom Number generator limitations
Manual state weighting and updating as per RicardoSantos's Function Markov Process is required:
What's being plotted for are the first three layers, but without the full range of the input_dimensions, hidden_dimensions:
syn_0 (blue)
syn_1 (green)
syn_h (red)
While there's more than a few technical hurdles to overcome (i.e. potential pine issues from max variables to runtime/compile limits, no real arrays, functions to do state updates RichardoSantos Markov Function style, etc), I'm fairly confident a limited working model should be possible to create in Pine.
