Debug into python system libraries in Visual Studio Code

By default, vscode does not let you to debug into python system libraries. If errors or exceptions happen in system libraries, you will only see the errors at the place you call the system library. Even if you set break points in system libraries, when it stops, it will stop at the place you call the system library. And the stack trace only shows the stack up to the call to system library. This can be very inconvenient sometimes. Fortunately, this behavior can be changed by setting "debugStdLib" to true in the launch configuration (can be opened through Menu->Debug->Open Configurations) like the following:

"configurations": [
{
    "name": "Python: Current File (Integrated Terminal)",
    "type": "python",
    "request": "launch",
    "program": "${file}",
    "console": "integratedTerminal",
    "debugStdLib": true,
},

Fixed randomness for tensorflow.keras for reproducible runs

Just follow the FAQ here
One of the steps is to set environment variable PYTHONHASHSEED=0 before starting python. For Visual Studio Code, you need to add the environment setting like the following in the configuration (can be opened through Menu->Debug->Open Configurations)

"configurations": [
{
    "name": "Python: Current File (Integrated Terminal)",
    "type": "python",
    "request": "launch",
    "program": "${file}",
    "console": "integratedTerminal",
    "debugStdLib": true,
    "env": {"PYTHONHASHSEED": 0}
},

Notes about Keras custum RNN

There are several limitations of keras.layers.RNN

1. It can have only one input.
2. It can have only one output.
3. The input has to be at least 3D (batch * timesteps * extra_dimensions). This is a slight inconvenience because you can use expand_dims to expand 2D tensor to 3D tensor.
4. The signature of cell.call() is not same as keras.Model.call(), this make it impossible to use Model as the base for cell. Thus it is very cumbersome to write complex custom cell consisting of many Layers.

Notes about Eager Execution using Keras

About model inputs
If the model is created using Sequential, the inputs can be numpy ndarray. However, if you compose the model using Keras layers like the MNISTModel in the Guide for Eager Execution, the inputs need to be Tensorflow Tensors. You can use tf.constant() to convert numpy arrays to Tensors. However, some types of numpy array are not accepted by tf.constant(). For example, 'a = np.random.randint(0, 10, (2,3)) % 5' does not work with tf.constant(). You need to do a type conversion a=a.astype(np.int32) before calling tf.constant.