Improvise a Jazz Solo with an LSTM Network

Welcome to your final programming assignment of this week! In this notebook, you will implement a model that uses an LSTM to generate music. You will even be able to listen to your own music at the end of the assignment.

You will learn to:

Apply an LSTM to music generation.

Generate your own jazz music with deep learning.

Please run the following cell to load all the packages required in this assignment. This may take a few minutes.

from __future__ import print_function
import IPython
import sys
from music21 import *
import numpy as np
from grammar import *
from qa import *
from preprocess import * 
from music_utils import *
from data_utils import *
from keras.models import load_model, Model
from keras.layers import Dense, Activation, Dropout, Input, LSTM, Reshape, Lambda, RepeatVector
from keras.initializers import glorot_uniform
from keras.utils import to_categorical
from keras.optimizers import Adam
from keras import backend as K

1 - Problem statement

You would like to create a jazz music piece specially for a friend's birthday. However, you don't know any instruments or music composition. Fortunately, you know deep learning and will solve this problem using an LSTM netwok.

You will train a network to generate novel jazz solos in a style representative of a body of performed work.

1.1 - Dataset

You will train your algorithm on a corpus of Jazz music. Run the cell below to listen to a snippet of the audio from the training set:

We have taken care of the preprocessing of the musical data to render it in terms of musical "values." You can informally think of each "value" as a note, which comprises a pitch and a duration. For example, if you press down a specific piano key for 0.5 seconds, then you have just played a note. In music theory, a "value" is actually more complicated than this--specifically, it also captures the information needed to play multiple notes at the same time. For example, when playing a music piece, you might press down two piano keys at the same time (playng multiple notes at the same time generates what's called a "chord"). But we don't need to worry about the details of music theory for this assignment. For the purpose of this assignment, all you need to know is that we will obtain a dataset of values, and will learn an RNN model to generate sequences of values.

Our music generation system will use 78 unique values. Run the following code to load the raw music data and preprocess it into values. This might take a few minutes.

X, Y, n_values, indices_values = load_music_utils()
print('shape of X:', X.shape)
print('number of training examples:', X.shape[0])
print('Tx (length of sequence):', X.shape[1])
print('total # of unique values:', n_values)
print('Shape of Y:', Y.shape)

You have just loaded the following:

X: This is an (m, Tx, 78) dimensional array. We have m training examples, each of which is a snippet of Tx=30 musical values. At each time step, the input is one of 78 different possible values, represented as a one-hot vector. Thus for example, X[i,t,:] is a one-hot vector representating the value of the i-th example at time t.
Y: This is essentially the same as X, but shifted one step to the left (to the past). Similar to the dinosaurus assignment, we're interested in the network using the previous values to predict the next value, so our sequence model will try to predict y⟨t⟩ given x⟨1⟩,…,x⟨t⟩. However, the data in Y is reordered to be dimension (Ty,m,78), where Ty=Tx. This format makes it more convenient to feed to the LSTM later.
n_values: The number of unique values in this dataset. This should be 78.
indices_values: python dictionary mapping from 0-77 to musical values.

1.2 - Overview of our model

Here is the architecture of the model we will use. This is similar to the Dinosaurus model you had used in the previous notebook, except that in you will be implementing it in Keras. The architecture is as follows:

We will be training the model on random snippets of 30 values taken from a much longer piece of music. Thus, we won't bother to set the first input

x⟨1⟩=0⃗ , which we had done previously to denote the start of a dinosaur name, since now most of these snippets of audio start somewhere in the middle of a piece of music. We are setting each of the snippts to have the same length Tx=30 to make vectorization easier.

2 - Building the model

In this part you will build and train a model that will learn musical patterns. To do so, you will need to build a model that takes in X of shape (m,Tx,78) and Y of shape (Ty,m,78). We will use an LSTM with 64 dimensional hidden states. Lets set n_a = 64.

Here's how you can create a Keras model with multiple inputs and outputs. If you're building an RNN where even at test time entire input sequence x⟨1⟩,x⟨2⟩,…,x⟨Tx⟩ were given in advance, for example if the inputs were words and the output was a label, then Keras has simple built-in functions to build the model. However, for sequence generation, at test time we don't know all the values of x⟨t⟩ in advance; instead we generate them one at a time using x⟨t⟩=y⟨t

吳恩達 Coursera Deep Learning 第五課 Sequence Models 第一週程式設計作業 3

Improvise a Jazz Solo with an LSTM Network

1 - Problem statement

1.1 - Dataset

1.2 - Overview of our model

2 - Building the model

吳恩達 Coursera Deep Learning 第五課 Sequence Models 第一週程式設計作業 3

Coursera Deep Learning 第四課卷積神經網路程式設計作業: Convolutional Model: Application

吳恩達Deeplearning.ai 第五課 Sequence Model 第一週------Deep RNNs

吳恩達Deeplearning.ai 第五課 Sequence Model 第一週------Sampling novel sequence

v2 吳恩達老師深度學習第五課第二週程式設計作業2

吳恩達Deeplearning.ai 第五課 Sequence Model 第一週------Recurrent Neural Network Model

吳恩達Deeplearning.ai 第五課 Sequence Model 第一週------Long Short Term Memory(LSTM)

吳恩達Deeplearning.ai 第五課 Sequence Model 第一週------Backpropagation through time

Coursera吳恩達機器學習課程-第五章

吳恩達機器學習（第五章）--特徵縮放和學習率

Coursera Deep Learning 第四課卷積神經網路第二週程式設計作業殘差神經網路 Residual Networks

Coursera 吳恩達 Deep Learning 第2課 Improving Deep Neural Networks 第一週程式設計作業程式碼 Regularization

Coursera 吳恩達 Deep Learning 第2課 Improving Deep Neural Networks 第一週程式設計作業程式碼 Initialization

Coursera-吳恩達-機器學習-（第5周筆記）Neural Networks——Learning

Coursera 吳恩達DeepLearning.AI 第五課 sequence model 序列模型第一週 Improvise a Jazz Solo with an LSTM Network

吳恩達-coursera-機器學習測試題第五週

【吳恩達 Coursera深度學習課程】 Neural Networks and Deep Learning 第一週課後習題

Coursera 吳恩達DeepLearning.AI 第五課 sequence model 序列模型第二週 Emofify

Coursera-AndrewNg(吳恩達)機器學習筆記——第三周

吳恩達機器學習（第十五章）---降維PCA

吳恩達 Coursera Deep Learning 第五課 Sequence Models 第一週程式設計作業 3

Improvise a Jazz Solo with an LSTM Network

1 - Problem statement

1.1 - Dataset

1.2 - Overview of our model

2 - Building the model

相關推薦