 TensorFlow.org에서 보기 |  Google Colab에서 실행 |  GitHub에서 소스 보기 |  노트북 다운로드 |
이 튜토리얼은 간단한 RNN을 사용하여 음표를 생성하는 방법을 보여줍니다. MAESTRO 데이터 세트 의 피아노 MIDI 파일 모음을 사용하여 모델을 훈련합니다. 일련의 음표가 주어지면 모델은 순서대로 다음 음표를 예측하는 방법을 학습합니다. 모델을 반복적으로 호출하여 더 긴 노트 시퀀스를 생성할 수 있습니다.
이 튜토리얼에는 MIDI 파일을 구문 분석하고 생성하기 위한 완전한 코드가 포함되어 있습니다. RNN으로 텍스트 생성을 방문하여 RNN의 작동 방식에 대해 자세히 알아볼 수 있습니다.
설정
이 튜토리얼에서는 pretty_midi 라이브러리를 사용하여 MIDI 파일을 만들고 구문 분석하고 pyfluidsynth 를 사용하여 Colab에서 오디오 재생을 생성합니다.
sudo apt install -y fluidsynth
The following packages were automatically installed and are no longer required: linux-gcp-5.4-headers-5.4.0-1040 linux-gcp-5.4-headers-5.4.0-1043 linux-gcp-5.4-headers-5.4.0-1044 linux-gcp-5.4-headers-5.4.0-1049 linux-headers-5.4.0-1049-gcp linux-image-5.4.0-1049-gcp linux-modules-5.4.0-1049-gcp linux-modules-extra-5.4.0-1049-gcp Use 'sudo apt autoremove' to remove them. The following additional packages will be installed: fluid-soundfont-gm libasyncns0 libdouble-conversion1 libevdev2 libflac8 libfluidsynth1 libgudev-1.0-0 libinput-bin libinput10 libjack-jackd2-0 libmtdev1 libogg0 libpulse0 libqt5core5a libqt5dbus5 libqt5gui5 libqt5network5 libqt5svg5 libqt5widgets5 libqt5x11extras5 libsamplerate0 libsndfile1 libvorbis0a libvorbisenc2 libwacom-bin libwacom-common libwacom2 libxcb-icccm4 libxcb-image0 libxcb-keysyms1 libxcb-randr0 libxcb-render-util0 libxcb-shape0 libxcb-util1 libxcb-xinerama0 libxcb-xkb1 libxkbcommon-x11-0 qsynth qt5-gtk-platformtheme qttranslations5-l10n Suggested packages: fluid-soundfont-gs timidity jackd2 pulseaudio qt5-image-formats-plugins qtwayland5 jackd The following NEW packages will be installed: fluid-soundfont-gm fluidsynth libasyncns0 libdouble-conversion1 libevdev2 libflac8 libfluidsynth1 libgudev-1.0-0 libinput-bin libinput10 libjack-jackd2-0 libmtdev1 libogg0 libpulse0 libqt5core5a libqt5dbus5 libqt5gui5 libqt5network5 libqt5svg5 libqt5widgets5 libqt5x11extras5 libsamplerate0 libsndfile1 libvorbis0a libvorbisenc2 libwacom-bin libwacom-common libwacom2 libxcb-icccm4 libxcb-image0 libxcb-keysyms1 libxcb-randr0 libxcb-render-util0 libxcb-shape0 libxcb-util1 libxcb-xinerama0 libxcb-xkb1 libxkbcommon-x11-0 qsynth qt5-gtk-platformtheme qttranslations5-l10n 0 upgraded, 41 newly installed, 0 to remove and 120 not upgraded. Need to get 132 MB of archives. After this operation, 198 MB of additional disk space will be used. Get:1 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libogg0 amd64 1.3.2-1 [17.2 kB] Get:2 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libdouble-conversion1 amd64 2.0.1-4ubuntu1 [33.0 kB] Get:3 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqt5core5a amd64 5.9.5+dfsg-0ubuntu2.6 [2035 kB] Get:4 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libevdev2 amd64 1.5.8+dfsg-1ubuntu0.1 [28.9 kB] Get:5 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libmtdev1 amd64 1.1.5-1ubuntu3 [13.8 kB] Get:6 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libgudev-1.0-0 amd64 1:232-2 [13.6 kB] Get:7 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libwacom-common all 0.29-1 [36.9 kB] Get:8 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libwacom2 amd64 0.29-1 [17.7 kB] Get:9 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libinput-bin amd64 1.10.4-1ubuntu0.18.04.2 [11.2 kB] Get:10 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libinput10 amd64 1.10.4-1ubuntu0.18.04.2 [86.2 kB] Get:11 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqt5dbus5 amd64 5.9.5+dfsg-0ubuntu2.6 [195 kB] Get:12 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqt5network5 amd64 5.9.5+dfsg-0ubuntu2.6 [634 kB] Get:13 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libxcb-icccm4 amd64 0.4.1-1ubuntu1 [10.4 kB] Get:14 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libxcb-util1 amd64 0.4.0-0ubuntu3 [11.2 kB] Get:15 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libxcb-image0 amd64 0.4.0-1build1 [12.3 kB] Get:16 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libxcb-keysyms1 amd64 0.4.0-1 [8406 B] Get:17 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxcb-randr0 amd64 1.13-2~ubuntu18.04 [16.4 kB] Get:18 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libxcb-render-util0 amd64 0.3.9-1 [9638 B] Get:19 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxcb-shape0 amd64 1.13-2~ubuntu18.04 [5972 B] Get:20 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxcb-xinerama0 amd64 1.13-2~ubuntu18.04 [5264 B] Get:21 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxcb-xkb1 amd64 1.13-2~ubuntu18.04 [30.1 kB] Get:22 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxkbcommon-x11-0 amd64 0.8.2-1~ubuntu18.04.1 [13.4 kB] Get:23 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqt5gui5 amd64 5.9.5+dfsg-0ubuntu2.6 [2568 kB] Get:24 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqt5widgets5 amd64 5.9.5+dfsg-0ubuntu2.6 [2203 kB] Get:25 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqt5svg5 amd64 5.9.5-0ubuntu1.1 [129 kB] Get:26 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/universe amd64 fluid-soundfont-gm all 3.1-5.1 [119 MB] Get:27 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libsamplerate0 amd64 0.1.9-1 [938 kB] Get:28 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libjack-jackd2-0 amd64 1.9.12~dfsg-2 [263 kB] Get:29 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libasyncns0 amd64 0.8-6 [12.1 kB] Get:30 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libflac8 amd64 1.3.2-1 [213 kB] Get:31 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libvorbis0a amd64 1.3.5-4.2 [86.4 kB] Get:32 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libvorbisenc2 amd64 1.3.5-4.2 [70.7 kB] Get:33 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libsndfile1 amd64 1.0.28-4ubuntu0.18.04.2 [170 kB] Get:34 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpulse0 amd64 1:11.1-1ubuntu7.11 [266 kB] Get:35 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/universe amd64 libfluidsynth1 amd64 1.1.9-1 [137 kB] Get:36 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/universe amd64 fluidsynth amd64 1.1.9-1 [20.7 kB] Get:37 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/universe amd64 libqt5x11extras5 amd64 5.9.5-0ubuntu1 [8596 B] Get:38 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libwacom-bin amd64 0.29-1 [4712 B] Get:39 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/universe amd64 qsynth amd64 0.5.0-2 [191 kB] Get:40 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 qt5-gtk-platformtheme amd64 5.9.5+dfsg-0ubuntu2.6 [117 kB] Get:41 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 qttranslations5-l10n all 5.9.5-0ubuntu1 [1485 kB] Fetched 132 MB in 9s (14.0 MB/s) Extracting templates from packages: 100% 7[0;23r8[1ASelecting previously unselected package libogg0:amd64. (Reading database ... 285125 files and directories currently installed.) Preparing to unpack .../00-libogg0_1.3.2-1_amd64.deb ... 7[24;0fProgress: [ 0%] [..........................................................] 8Unpacking libogg0:amd64 (1.3.2-1) ... 7[24;0fProgress: [ 1%] [..........................................................] 8Selecting previously unselected package libdouble-conversion1:amd64. Preparing to unpack .../01-libdouble-conversion1_2.0.1-4ubuntu1_amd64.deb ... Unpacking libdouble-conversion1:amd64 (2.0.1-4ubuntu1) ... 7[24;0fProgress: [ 2%] [#.........................................................] 8Selecting previously unselected package libqt5core5a:amd64. Preparing to unpack .../02-libqt5core5a_5.9.5+dfsg-0ubuntu2.6_amd64.deb ... 7[24;0fProgress: [ 3%] [#.........................................................] 8Unpacking libqt5core5a:amd64 (5.9.5+dfsg-0ubuntu2.6) ... 7[24;0fProgress: [ 4%] [##........................................................] 8Selecting previously unselected package libevdev2:amd64. Preparing to unpack .../03-libevdev2_1.5.8+dfsg-1ubuntu0.1_amd64.deb ... Unpacking libevdev2:amd64 (1.5.8+dfsg-1ubuntu0.1) ... 7[24;0fProgress: [ 5%] [###.......................................................] 8Selecting previously unselected package libmtdev1:amd64. Preparing to unpack .../04-libmtdev1_1.1.5-1ubuntu3_amd64.deb ... 7[24;0fProgress: [ 6%] [###.......................................................] 8Unpacking libmtdev1:amd64 (1.1.5-1ubuntu3) ... 7[24;0fProgress: [ 7%] [####......................................................] 8Selecting previously unselected package libgudev-1.0-0:amd64. Preparing to unpack .../05-libgudev-1.0-0_1%3a232-2_amd64.deb ... Unpacking libgudev-1.0-0:amd64 (1:232-2) ... 7[24;0fProgress: [ 8%] [####......................................................] 8Selecting previously unselected package libwacom-common. Preparing to unpack .../06-libwacom-common_0.29-1_all.deb ... 7[24;0fProgress: [ 9%] [#####.....................................................] 8Unpacking libwacom-common (0.29-1) ... 7[24;0fProgress: [ 10%] [#####.....................................................] 8Selecting previously unselected package libwacom2:amd64. Preparing to unpack .../07-libwacom2_0.29-1_amd64.deb ... Unpacking libwacom2:amd64 (0.29-1) ... 7[24;0fProgress: [ 11%] [######....................................................] 8Selecting previously unselected package libinput-bin. Preparing to unpack .../08-libinput-bin_1.10.4-1ubuntu0.18.04.2_amd64.deb ... 7[24;0fProgress: [ 12%] [#######...................................................] 8Unpacking libinput-bin (1.10.4-1ubuntu0.18.04.2) ... 7[24;0fProgress: [ 13%] [#######...................................................] 8Selecting previously unselected package libinput10:amd64. Preparing to unpack .../09-libinput10_1.10.4-1ubuntu0.18.04.2_amd64.deb ... Unpacking libinput10:amd64 (1.10.4-1ubuntu0.18.04.2) ... 7[24;0fProgress: [ 14%] [########..................................................] 8Selecting previously unselected package libqt5dbus5:amd64. Preparing to unpack .../10-libqt5dbus5_5.9.5+dfsg-0ubuntu2.6_amd64.deb ... 7[24;0fProgress: [ 15%] [########..................................................] 8Unpacking libqt5dbus5:amd64 (5.9.5+dfsg-0ubuntu2.6) ... 7[24;0fProgress: [ 16%] [#########.................................................] 8Selecting previously unselected package libqt5network5:amd64. Preparing to unpack .../11-libqt5network5_5.9.5+dfsg-0ubuntu2.6_amd64.deb ... Unpacking libqt5network5:amd64 (5.9.5+dfsg-0ubuntu2.6) ... 7[24;0fProgress: [ 17%] [##########................................................] 8Selecting previously unselected package libxcb-icccm4:amd64. Preparing to unpack .../12-libxcb-icccm4_0.4.1-1ubuntu1_amd64.deb ... Unpacking libxcb-icccm4:amd64 (0.4.1-1ubuntu1) ... 7[24;0fProgress: [ 18%] [##########................................................] 8Selecting previously unselected package libxcb-util1:amd64. Preparing to unpack .../13-libxcb-util1_0.4.0-0ubuntu3_amd64.deb ... 7[24;0fProgress: [ 19%] [###########...............................................] 8Unpacking libxcb-util1:amd64 (0.4.0-0ubuntu3) ... 7[24;0fProgress: [ 20%] [###########...............................................] 8Selecting previously unselected package libxcb-image0:amd64. Preparing to unpack .../14-libxcb-image0_0.4.0-1build1_amd64.deb ... Unpacking libxcb-image0:amd64 (0.4.0-1build1) ... 7[24;0fProgress: [ 21%] [############..............................................] 8Selecting previously unselected package libxcb-keysyms1:amd64. Preparing to unpack .../15-libxcb-keysyms1_0.4.0-1_amd64.deb ... 7[24;0fProgress: [ 22%] [############..............................................] 8Unpacking libxcb-keysyms1:amd64 (0.4.0-1) ... 7[24;0fProgress: [ 23%] [#############.............................................] 8Selecting previously unselected package libxcb-randr0:amd64. Preparing to unpack .../16-libxcb-randr0_1.13-2~ubuntu18.04_amd64.deb ... Unpacking libxcb-randr0:amd64 (1.13-2~ubuntu18.04) ... 7[24;0fProgress: [ 24%] [##############............................................] 8Selecting previously unselected package libxcb-render-util0:amd64. Preparing to unpack .../17-libxcb-render-util0_0.3.9-1_amd64.deb ... 7[24;0fProgress: [ 25%] [##############............................................] 8Unpacking libxcb-render-util0:amd64 (0.3.9-1) ... 7[24;0fProgress: [ 26%] [###############...........................................] 8Selecting previously unselected package libxcb-shape0:amd64. Preparing to unpack .../18-libxcb-shape0_1.13-2~ubuntu18.04_amd64.deb ... Unpacking libxcb-shape0:amd64 (1.13-2~ubuntu18.04) ... 7[24;0fProgress: [ 27%] [###############...........................................] 8Selecting previously unselected package libxcb-xinerama0:amd64. Preparing to unpack .../19-libxcb-xinerama0_1.13-2~ubuntu18.04_amd64.deb ... 7[24;0fProgress: [ 28%] [################..........................................] 8Unpacking libxcb-xinerama0:amd64 (1.13-2~ubuntu18.04) ... 7[24;0fProgress: [ 29%] [################..........................................] 8Selecting previously unselected package libxcb-xkb1:amd64. Preparing to unpack .../20-libxcb-xkb1_1.13-2~ubuntu18.04_amd64.deb ... Unpacking libxcb-xkb1:amd64 (1.13-2~ubuntu18.04) ... 7[24;0fProgress: [ 30%] [#################.........................................] 8Selecting previously unselected package libxkbcommon-x11-0:amd64. Preparing to unpack .../21-libxkbcommon-x11-0_0.8.2-1~ubuntu18.04.1_amd64.deb ... 7[24;0fProgress: [ 31%] [##################........................................] 8Unpacking libxkbcommon-x11-0:amd64 (0.8.2-1~ubuntu18.04.1) ... 7[24;0fProgress: [ 32%] [##################........................................] 8Selecting previously unselected package libqt5gui5:amd64. Preparing to unpack .../22-libqt5gui5_5.9.5+dfsg-0ubuntu2.6_amd64.deb ... Unpacking libqt5gui5:amd64 (5.9.5+dfsg-0ubuntu2.6) ... 7[24;0fProgress: [ 33%] [###################.......................................] 8Selecting previously unselected package libqt5widgets5:amd64. Preparing to unpack .../23-libqt5widgets5_5.9.5+dfsg-0ubuntu2.6_amd64.deb ... Unpacking libqt5widgets5:amd64 (5.9.5+dfsg-0ubuntu2.6) ... 7[24;0fProgress: [ 34%] [###################.......................................] 8Selecting previously unselected package libqt5svg5:amd64. Preparing to unpack .../24-libqt5svg5_5.9.5-0ubuntu1.1_amd64.deb ... 7[24;0fProgress: [ 35%] [####################......................................] 8Unpacking libqt5svg5:amd64 (5.9.5-0ubuntu1.1) ... 7[24;0fProgress: [ 36%] [#####################.....................................] 8Selecting previously unselected package fluid-soundfont-gm. Preparing to unpack .../25-fluid-soundfont-gm_3.1-5.1_all.deb ... Unpacking fluid-soundfont-gm (3.1-5.1) ... 7[24;0fProgress: [ 37%] [#####################.....................................] 8Selecting previously unselected package libsamplerate0:amd64. Preparing to unpack .../26-libsamplerate0_0.1.9-1_amd64.deb ... 7[24;0fProgress: [ 38%] [######################....................................] 8Unpacking libsamplerate0:amd64 (0.1.9-1) ... 7[24;0fProgress: [ 39%] [######################....................................] 8Selecting previously unselected package libjack-jackd2-0:amd64. Preparing to unpack .../27-libjack-jackd2-0_1.9.12~dfsg-2_amd64.deb ... Unpacking libjack-jackd2-0:amd64 (1.9.12~dfsg-2) ... 7[24;0fProgress: [ 40%] [#######################...................................] 8Selecting previously unselected package libasyncns0:amd64. Preparing to unpack .../28-libasyncns0_0.8-6_amd64.deb ... 7[24;0fProgress: [ 41%] [#######################...................................] 8Unpacking libasyncns0:amd64 (0.8-6) ... 7[24;0fProgress: [ 42%] [########################..................................] 8Selecting previously unselected package libflac8:amd64. Preparing to unpack .../29-libflac8_1.3.2-1_amd64.deb ... Unpacking libflac8:amd64 (1.3.2-1) ... 7[24;0fProgress: [ 43%] [#########################.................................] 8Selecting previously unselected package libvorbis0a:amd64. Preparing to unpack .../30-libvorbis0a_1.3.5-4.2_amd64.deb ... 7[24;0fProgress: [ 44%] [#########################.................................] 8Unpacking libvorbis0a:amd64 (1.3.5-4.2) ... 7[24;0fProgress: [ 45%] [##########################................................] 8Selecting previously unselected package libvorbisenc2:amd64. Preparing to unpack .../31-libvorbisenc2_1.3.5-4.2_amd64.deb ... Unpacking libvorbisenc2:amd64 (1.3.5-4.2) ... 7[24;0fProgress: [ 46%] [##########################................................] 8Selecting previously unselected package libsndfile1:amd64. Preparing to unpack .../32-libsndfile1_1.0.28-4ubuntu0.18.04.2_amd64.deb ... 7[24;0fProgress: [ 47%] [###########################...............................] 8Unpacking libsndfile1:amd64 (1.0.28-4ubuntu0.18.04.2) ... 7[24;0fProgress: [ 48%] [###########################...............................] 8Selecting previously unselected package libpulse0:amd64. Preparing to unpack .../33-libpulse0_1%3a11.1-1ubuntu7.11_amd64.deb ... Unpacking libpulse0:amd64 (1:11.1-1ubuntu7.11) ... 7[24;0fProgress: [ 49%] [############################..............................] 8Selecting previously unselected package libfluidsynth1:amd64. Preparing to unpack .../34-libfluidsynth1_1.1.9-1_amd64.deb ... 7[24;0fProgress: [ 50%] [#############################.............................] 8Unpacking libfluidsynth1:amd64 (1.1.9-1) ... Selecting previously unselected package fluidsynth. Preparing to unpack .../35-fluidsynth_1.1.9-1_amd64.deb ... 7[24;0fProgress: [ 51%] [#############################.............................] 8Unpacking fluidsynth (1.1.9-1) ... 7[24;0fProgress: [ 52%] [##############################............................] 8Selecting previously unselected package libqt5x11extras5:amd64. Preparing to unpack .../36-libqt5x11extras5_5.9.5-0ubuntu1_amd64.deb ... Unpacking libqt5x11extras5:amd64 (5.9.5-0ubuntu1) ... 7[24;0fProgress: [ 53%] [##############################............................] 8Selecting previously unselected package libwacom-bin. Preparing to unpack .../37-libwacom-bin_0.29-1_amd64.deb ... 7[24;0fProgress: [ 54%] [###############################...........................] 8Unpacking libwacom-bin (0.29-1) ... 7[24;0fProgress: [ 55%] [################################..........................] 8Selecting previously unselected package qsynth. Preparing to unpack .../38-qsynth_0.5.0-2_amd64.deb ... Unpacking qsynth (0.5.0-2) ... 7[24;0fProgress: [ 56%] [################################..........................] 8Selecting previously unselected package qt5-gtk-platformtheme:amd64. Preparing to unpack .../39-qt5-gtk-platformtheme_5.9.5+dfsg-0ubuntu2.6_amd64.deb ... 7[24;0fProgress: [ 57%] [#################################.........................] 8Unpacking qt5-gtk-platformtheme:amd64 (5.9.5+dfsg-0ubuntu2.6) ... 7[24;0fProgress: [ 58%] [#################################.........................] 8Selecting previously unselected package qttranslations5-l10n. Preparing to unpack .../40-qttranslations5-l10n_5.9.5-0ubuntu1_all.deb ... Unpacking qttranslations5-l10n (5.9.5-0ubuntu1) ... 7[24;0fProgress: [ 59%] [##################################........................] 8Setting up libxcb-xinerama0:amd64 (1.13-2~ubuntu18.04) ... 7[24;0fProgress: [ 60%] [##################################........................] 8Setting up libxcb-render-util0:amd64 (0.3.9-1) ... 7[24;0fProgress: [ 61%] [###################################.......................] 8Setting up libxcb-randr0:amd64 (1.13-2~ubuntu18.04) ... 7[24;0fProgress: [ 62%] [####################################......................] 8Setting up libxcb-icccm4:amd64 (0.4.1-1ubuntu1) ... 7[24;0fProgress: [ 63%] [####################################......................] 8Setting up libasyncns0:amd64 (0.8-6) ... 7[24;0fProgress: [ 64%] [#####################################.....................] 8Setting up libwacom-common (0.29-1) ... 7[24;0fProgress: [ 65%] [#####################################.....................] 8Setting up libdouble-conversion1:amd64 (2.0.1-4ubuntu1) ... 7[24;0fProgress: [ 66%] [######################################....................] 8Setting up libevdev2:amd64 (1.5.8+dfsg-1ubuntu0.1) ... 7[24;0fProgress: [ 67%] [#######################################...................] 8Setting up fluid-soundfont-gm (3.1-5.1) ... 7[24;0fProgress: [ 68%] [#######################################...................] 8Setting up libxcb-util1:amd64 (0.4.0-0ubuntu3) ... 7[24;0fProgress: [ 69%] [########################################..................] 8Setting up libogg0:amd64 (1.3.2-1) ... 7[24;0fProgress: [ 70%] [########################################..................] 8Setting up qttranslations5-l10n (5.9.5-0ubuntu1) ... 7[24;0fProgress: [ 71%] [#########################################.................] 8Setting up libmtdev1:amd64 (1.1.5-1ubuntu3) ... 7[24;0fProgress: [ 72%] [#########################################.................] 8Setting up libxcb-shape0:amd64 (1.13-2~ubuntu18.04) ... 7[24;0fProgress: [ 73%] [##########################################................] 8Setting up libgudev-1.0-0:amd64 (1:232-2) ... 7[24;0fProgress: [ 74%] [###########################################...............] 8Setting up libxcb-keysyms1:amd64 (0.4.0-1) ... 7[24;0fProgress: [ 75%] [###########################################...............] 8Setting up libsamplerate0:amd64 (0.1.9-1) ... 7[24;0fProgress: [ 76%] [############################################..............] 8Setting up libvorbis0a:amd64 (1.3.5-4.2) ... 7[24;0fProgress: [ 77%] [############################################..............] 8Setting up libxcb-xkb1:amd64 (1.13-2~ubuntu18.04) ... 7[24;0fProgress: [ 78%] [#############################################.............] 8Setting up libqt5core5a:amd64 (5.9.5+dfsg-0ubuntu2.6) ... 7[24;0fProgress: [ 79%] [#############################################.............] 8Setting up libqt5dbus5:amd64 (5.9.5+dfsg-0ubuntu2.6) ... 7[24;0fProgress: [ 80%] [##############################################............] 8Setting up libqt5network5:amd64 (5.9.5+dfsg-0ubuntu2.6) ... 7[24;0fProgress: [ 81%] [###############################################...........] 8Setting up libwacom2:amd64 (0.29-1) ... 7[24;0fProgress: [ 82%] [###############################################...........] 8Setting up libxcb-image0:amd64 (0.4.0-1build1) ... 7[24;0fProgress: [ 83%] [################################################..........] 8Setting up libflac8:amd64 (1.3.2-1) ... Setting up libinput-bin (1.10.4-1ubuntu0.18.04.2) ... 7[24;0fProgress: [ 84%] [################################################..........] 8Setting up libxkbcommon-x11-0:amd64 (0.8.2-1~ubuntu18.04.1) ... 7[24;0fProgress: [ 85%] [#################################################.........] 8Setting up libwacom-bin (0.29-1) ... 7[24;0fProgress: [ 86%] [##################################################........] 8Setting up libjack-jackd2-0:amd64 (1.9.12~dfsg-2) ... 7[24;0fProgress: [ 87%] [##################################################........] 8Setting up libvorbisenc2:amd64 (1.3.5-4.2) ... 7[24;0fProgress: [ 88%] [###################################################.......] 8Setting up libinput10:amd64 (1.10.4-1ubuntu0.18.04.2) ... 7[24;0fProgress: [ 89%] [###################################################.......] 8Setting up libsndfile1:amd64 (1.0.28-4ubuntu0.18.04.2) ... 7[24;0fProgress: [ 90%] [####################################################......] 8Setting up libqt5gui5:amd64 (5.9.5+dfsg-0ubuntu2.6) ... 7[24;0fProgress: [ 91%] [####################################################......] 8Setting up qt5-gtk-platformtheme:amd64 (5.9.5+dfsg-0ubuntu2.6) ... 7[24;0fProgress: [ 92%] [#####################################################.....] 8Setting up libqt5x11extras5:amd64 (5.9.5-0ubuntu1) ... 7[24;0fProgress: [ 93%] [######################################################....] 8Setting up libqt5widgets5:amd64 (5.9.5+dfsg-0ubuntu2.6) ... 7[24;0fProgress: [ 94%] [######################################################....] 8Setting up libpulse0:amd64 (1:11.1-1ubuntu7.11) ... 7[24;0fProgress: [ 95%] [#######################################################...] 8Setting up libqt5svg5:amd64 (5.9.5-0ubuntu1.1) ... 7[24;0fProgress: [ 96%] [#######################################################...] 8Setting up libfluidsynth1:amd64 (1.1.9-1) ... 7[24;0fProgress: [ 97%] [########################################################..] 8Setting up fluidsynth (1.1.9-1) ... 7[24;0fProgress: [ 98%] [########################################################..] 8Setting up qsynth (0.5.0-2) ... 7[24;0fProgress: [ 99%] [#########################################################.] 8Processing triggers for hicolor-icon-theme (0.17-2) ... Processing triggers for mime-support (3.60ubuntu1) ... Processing triggers for libc-bin (2.27-3ubuntu1.2) ... Processing triggers for udev (237-3ubuntu10.50) ... Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 7[0;24r8[1A[J
pip install --upgrade pyfluidsynth
pip install pretty_midi
import collections
import datetime
import fluidsynth
import glob
import numpy as np
import pathlib
import pandas as pd
import pretty_midi
import seaborn as sns
import tensorflow as tf
from IPython import display
from matplotlib import pyplot as plt
from typing import Dict, List, Optional, Sequence, Tuple
seed = 42
tf.random.set_seed(seed)
np.random.seed(seed)
# Sampling rate for audio playback
_SAMPLING_RATE = 16000
Maestro 데이터 세트 다운로드
data_dir = pathlib.Path('data/maestro-v2.0.0')
if not data_dir.exists():
tf.keras.utils.get_file(
'maestro-v2.0.0-midi.zip',
origin='https://storage.googleapis.com/magentadata/datasets/maestro/v2.0.0/maestro-v2.0.0-midi.zip',
extract=True,
cache_dir='.', cache_subdir='data',
)
Downloading data from https://storage.googleapis.com/magentadata/datasets/maestro/v2.0.0/maestro-v2.0.0-midi.zip 59244544/59243107 [==============================] - 3s 0us/step 59252736/59243107 [==============================] - 3s 0us/step
데이터 세트에는 약 1,200개의 MIDI 파일이 포함되어 있습니다.
filenames = glob.glob(str(data_dir/'**/*.mid*'))
print('Number of files:', len(filenames))
Number of files: 1282
MIDI 파일 처리
먼저, pretty_midi 를 사용하여 단일 MIDI 파일을 구문 분석하고 음표 형식을 검사합니다. 아래 MIDI 파일을 다운로드하여 컴퓨터에서 재생하려면 colab에서 files.download(sample_file) 을 작성하여 다운로드할 수 있습니다.
sample_file = filenames[1]
print(sample_file)
data/maestro-v2.0.0/2013/ORIG-MIDI_02_7_6_13_Group__MID--AUDIO_08_R1_2013_wav--3.midi
샘플 MIDI 파일에 대한 PrettyMIDI 개체를 생성합니다.
pm = pretty_midi.PrettyMIDI(sample_file)
샘플 파일을 재생합니다. 재생 위젯을 로드하는 데 몇 초가 걸릴 수 있습니다.
def display_audio(pm: pretty_midi.PrettyMIDI, seconds=30):
waveform = pm.fluidsynth(fs=_SAMPLING_RATE)
# Take a sample of the generated waveform to mitigate kernel resets
waveform_short = waveform[:seconds*_SAMPLING_RATE]
return display.Audio(waveform_short, rate=_SAMPLING_RATE)
display_audio(pm)
MIDI 파일을 검사하십시오. 어떤 종류의 악기가 사용됩니까?
print('Number of instruments:', len(pm.instruments))
instrument = pm.instruments[0]
instrument_name = pretty_midi.program_to_instrument_name(instrument.program)
print('Instrument name:', instrument_name)
Number of instruments: 1 Instrument name: Acoustic Grand Piano
메모 추출
for i, note in enumerate(instrument.notes[:10]):
note_name = pretty_midi.note_number_to_name(note.pitch)
duration = note.end - note.start
print(f'{i}: pitch={note.pitch}, note_name={note_name},'
f' duration={duration:.4f}')
0: pitch=56, note_name=G#3, duration=0.0352 1: pitch=44, note_name=G#2, duration=0.0417 2: pitch=68, note_name=G#4, duration=0.0651 3: pitch=80, note_name=G#5, duration=0.1693 4: pitch=78, note_name=F#5, duration=0.1523 5: pitch=76, note_name=E5, duration=0.1120 6: pitch=75, note_name=D#5, duration=0.0612 7: pitch=49, note_name=C#3, duration=0.0378 8: pitch=85, note_name=C#6, duration=0.0352 9: pitch=37, note_name=C#2, duration=0.0417
모델을 훈련할 때 음표를 나타내기 위해 세 가지 변수( pitch , step 및 duration )를 사용합니다. 피치는 MIDI 음표 번호로서의 사운드의 지각적 품질입니다. step 는 트랙의 이전 음표 또는 시작 부분에서 경과된 시간입니다. duration 은 음표가 재생되는 시간(초)이며 음표 종료 시간과 음표 시작 시간의 차이입니다.
샘플 MIDI 파일에서 음표를 추출합니다.
def midi_to_notes(midi_file: str) -> pd.DataFrame:
pm = pretty_midi.PrettyMIDI(midi_file)
instrument = pm.instruments[0]
notes = collections.defaultdict(list)
# Sort the notes by start time
sorted_notes = sorted(instrument.notes, key=lambda note: note.start)
prev_start = sorted_notes[0].start
for note in sorted_notes:
start = note.start
end = note.end
notes['pitch'].append(note.pitch)
notes['start'].append(start)
notes['end'].append(end)
notes['step'].append(start - prev_start)
notes['duration'].append(end - start)
prev_start = start
return pd.DataFrame({name: np.array(value) for name, value in notes.items()})
raw_notes = midi_to_notes(sample_file)
raw_notes.head()
음정보다 음표 이름을 해석하는 것이 더 쉬울 수 있으므로 아래 기능을 사용하여 숫자 음높이 값에서 음표 이름으로 변환할 수 있습니다. 음표 이름은 음표 유형, 우발적 및 옥타브 번호(예: C#4)를 나타냅니다.
get_note_names = np.vectorize(pretty_midi.note_number_to_name)
sample_note_names = get_note_names(raw_notes['pitch'])
sample_note_names[:10]
array(['G#3', 'G#5', 'G#4', 'G#2', 'F#5', 'E5', 'D#5', 'C#3', 'C#6',
'C#5'], dtype='<U3')
악곡을 시각화하려면 트랙 길이(즉, 피아노 롤)에 걸쳐 음표 피치, 시작 및 끝을 플롯합니다. 처음 100개의 메모로 시작
def plot_piano_roll(notes: pd.DataFrame, count: Optional[int] = None):
if count:
title = f'First {count} notes'
else:
title = f'Whole track'
count = len(notes['pitch'])
plt.figure(figsize=(20, 4))
plot_pitch = np.stack([notes['pitch'], notes['pitch']], axis=0)
plot_start_stop = np.stack([notes['start'], notes['end']], axis=0)
plt.plot(
plot_start_stop[:, :count], plot_pitch[:, :count], color="b", marker=".")
plt.xlabel('Time [s]')
plt.ylabel('Pitch')
_ = plt.title(title)
plot_piano_roll(raw_notes, count=100)
전체 트랙에 대한 메모를 플로팅합니다.
plot_piano_roll(raw_notes)
각 음표 변수의 분포를 확인하십시오.
def plot_distributions(notes: pd.DataFrame, drop_percentile=2.5):
plt.figure(figsize=[15, 5])
plt.subplot(1, 3, 1)
sns.histplot(notes, x="pitch", bins=20)
plt.subplot(1, 3, 2)
max_step = np.percentile(notes['step'], 100 - drop_percentile)
sns.histplot(notes, x="step", bins=np.linspace(0, max_step, 21))
plt.subplot(1, 3, 3)
max_duration = np.percentile(notes['duration'], 100 - drop_percentile)
sns.histplot(notes, x="duration", bins=np.linspace(0, max_duration, 21))
plot_distributions(raw_notes)
MIDI 파일 생성
아래 기능을 사용하여 음표 목록에서 자신의 MIDI 파일을 생성할 수 있습니다.
def notes_to_midi(
notes: pd.DataFrame,
out_file: str,
instrument_name: str,
velocity: int = 100, # note loudness
) -> pretty_midi.PrettyMIDI:
pm = pretty_midi.PrettyMIDI()
instrument = pretty_midi.Instrument(
program=pretty_midi.instrument_name_to_program(
instrument_name))
prev_start = 0
for i, note in notes.iterrows():
start = float(prev_start + note['step'])
end = float(start + note['duration'])
note = pretty_midi.Note(
velocity=velocity,
pitch=int(note['pitch']),
start=start,
end=end,
)
instrument.notes.append(note)
prev_start = start
pm.instruments.append(instrument)
pm.write(out_file)
return pm
example_file = 'example.midi'
example_pm = notes_to_midi(
raw_notes, out_file=example_file, instrument_name=instrument_name)
생성된 MIDI 파일을 재생하여 차이점이 있는지 확인합니다.
display_audio(example_pm)
이전과 마찬가지로 files.download(example_file) 를 작성하여 이 파일을 다운로드하고 재생할 수 있습니다.
훈련 데이터 세트 생성
MIDI 파일에서 메모를 추출하여 교육 데이터 세트를 만듭니다. 적은 수의 파일을 사용하여 시작하고 나중에 더 많은 파일로 실험할 수 있습니다. 몇 분 정도 걸릴 수 있습니다.
num_files = 5
all_notes = []
for f in filenames[:num_files]:
notes = midi_to_notes(f)
all_notes.append(notes)
all_notes = pd.concat(all_notes)
n_notes = len(all_notes)
print('Number of notes parsed:', n_notes)
Number of notes parsed: 23163
다음으로, 구문 분석된 메모에서 tf.data.Dataset 을 만듭니다.
key_order = ['pitch', 'step', 'duration']
train_notes = np.stack([all_notes[key] for key in key_order], axis=1)
notes_ds = tf.data.Dataset.from_tensor_slices(train_notes)
notes_ds.element_spec
TensorSpec(shape=(3,), dtype=tf.float64, name=None)
일련의 노트에 대해 모델을 훈련할 것입니다. 각 예제는 입력 기능으로 일련의 메모와 레이블로 다음 메모로 구성됩니다. 이러한 방식으로 모델은 시퀀스의 다음 음표를 예측하도록 훈련됩니다. RNN을 사용한 텍스트 분류 에서 이 프로세스(및 자세한 내용)를 설명하는 다이어그램을 찾을 수 있습니다.
크기가 seq_length 인 편리한 창 기능을 사용하여 이 형식으로 기능과 레이블을 생성할 수 있습니다.
def create_sequences(
dataset: tf.data.Dataset,
seq_length: int,
vocab_size = 128,
) -> tf.data.Dataset:
"""Returns TF Dataset of sequence and label examples."""
seq_length = seq_length+1
# Take 1 extra for the labels
windows = dataset.window(seq_length, shift=1, stride=1,
drop_remainder=True)
# `flat_map` flattens the" dataset of datasets" into a dataset of tensors
flatten = lambda x: x.batch(seq_length, drop_remainder=True)
sequences = windows.flat_map(flatten)
# Normalize note pitch
def scale_pitch(x):
x = x/[vocab_size,1.0,1.0]
return x
# Split the labels
def split_labels(sequences):
inputs = sequences[:-1]
labels_dense = sequences[-1]
labels = {key:labels_dense[i] for i,key in enumerate(key_order)}
return scale_pitch(inputs), labels
return sequences.map(split_labels, num_parallel_calls=tf.data.AUTOTUNE)
각 예의 시퀀스 길이를 설정합니다. 다른 길이(예: 50, 100, 150)로 실험하여 어느 것이 데이터에 가장 적합한지 확인하거나 초매개변수 조정 을 사용합니다. 어휘( vocab_size )의 크기는 pretty_midi 가 지원하는 모든 피치를 나타내는 128로 설정됩니다.
seq_length = 25
vocab_size = 128
seq_ds = create_sequences(notes_ds, seq_length, vocab_size)
seq_ds.element_spec
(TensorSpec(shape=(25, 3), dtype=tf.float64, name=None),
{'pitch': TensorSpec(shape=(), dtype=tf.float64, name=None),
'step': TensorSpec(shape=(), dtype=tf.float64, name=None),
'duration': TensorSpec(shape=(), dtype=tf.float64, name=None)})
데이터 세트의 모양은 (100,1) 입니다. 즉, 모델은 100개의 메모를 입력으로 사용하고 다음 메모를 출력으로 예측하는 방법을 학습합니다.
for seq, target in seq_ds.take(1):
print('sequence shape:', seq.shape)
print('sequence elements (first 10):', seq[0: 10])
print()
print('target:', target)
sequence shape: (25, 3)
sequence elements (first 10): tf.Tensor(
[[0.578125 0. 0.1484375 ]
[0.390625 0.00130208 0.0390625 ]
[0.3828125 0.03255208 0.07421875]
[0.390625 0.08203125 0.14713542]
[0.5625 0.14973958 0.07421875]
[0.546875 0.09375 0.07421875]
[0.5390625 0.12239583 0.04947917]
[0.296875 0.01692708 0.31119792]
[0.5234375 0.09895833 0.04036458]
[0.5078125 0.12369792 0.06380208]], shape=(10, 3), dtype=float64)
target: {'pitch': <tf.Tensor: shape=(), dtype=float64, numpy=67.0>, 'step': <tf.Tensor: shape=(), dtype=float64, numpy=0.1171875>, 'duration': <tf.Tensor: shape=(), dtype=float64, numpy=0.04947916666666652>}
예제를 일괄 처리하고 성능을 위해 데이터 세트를 구성합니다.
batch_size = 64
buffer_size = n_notes - seq_length # the number of items in the dataset
train_ds = (seq_ds
.shuffle(buffer_size)
.batch(batch_size, drop_remainder=True)
.cache()
.prefetch(tf.data.experimental.AUTOTUNE))
train_ds.element_spec
(TensorSpec(shape=(64, 25, 3), dtype=tf.float64, name=None),
{'pitch': TensorSpec(shape=(64,), dtype=tf.float64, name=None),
'step': TensorSpec(shape=(64,), dtype=tf.float64, name=None),
'duration': TensorSpec(shape=(64,), dtype=tf.float64, name=None)})
모델 생성 및 학습
모델에는 각 음 변수에 대해 하나씩 3개의 출력이 있습니다. pitch 및 duration 의 경우 모델이 음이 아닌 값을 출력하도록 권장하는 평균 제곱 오차를 기반으로 하는 사용자 정의 손실 함수를 사용합니다.
def mse_with_positive_pressure(y_true: tf.Tensor, y_pred: tf.Tensor):
mse = (y_true - y_pred) ** 2
positive_pressure = 10 * tf.maximum(-y_pred, 0.0)
return tf.reduce_mean(mse + positive_pressure)
input_shape = (seq_length, 3)
learning_rate = 0.005
inputs = tf.keras.Input(input_shape)
x = tf.keras.layers.LSTM(128)(inputs)
outputs = {
'pitch': tf.keras.layers.Dense(128, name='pitch')(x),
'step': tf.keras.layers.Dense(1, name='step')(x),
'duration': tf.keras.layers.Dense(1, name='duration')(x),
}
model = tf.keras.Model(inputs, outputs)
loss = {
'pitch': tf.keras.losses.SparseCategoricalCrossentropy(
from_logits=True),
'step': mse_with_positive_pressure,
'duration': mse_with_positive_pressure,
}
optimizer = tf.keras.optimizers.Adam(learning_rate=learning_rate)
model.compile(loss=loss, optimizer=optimizer)
model.summary()
Model: "model"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, 25, 3)] 0 []
lstm (LSTM) (None, 128) 67584 ['input_1[0][0]']
duration (Dense) (None, 1) 129 ['lstm[0][0]']
pitch (Dense) (None, 128) 16512 ['lstm[0][0]']
step (Dense) (None, 1) 129 ['lstm[0][0]']
==================================================================================================
Total params: 84,354
Trainable params: 84,354
Non-trainable params: 0
__________________________________________________________________________________________________
model.evaluate 함수를 테스트하면 pitch 손실이 step 및 duration 손실보다 훨씬 큰 것을 알 수 있습니다. loss 은 다른 모든 손실을 합산하여 계산된 총 손실이며 현재 pitch 손실이 지배합니다.
losses = model.evaluate(train_ds, return_dict=True)
losses
361/361 [==============================] - 6s 4ms/step - loss: 5.0011 - duration_loss: 0.1213 - pitch_loss: 4.8476 - step_loss: 0.0322
{'loss': 5.001128196716309,
'duration_loss': 0.12134315073490143,
'pitch_loss': 4.847629547119141,
'step_loss': 0.03215572610497475}
이 균형을 유지하는 한 가지 방법은 loss_weights 인수를 사용하여 컴파일하는 것입니다.
model.compile(
loss=loss,
loss_weights={
'pitch': 0.05,
'step': 1.0,
'duration':1.0,
},
optimizer=optimizer,
)
그런 다음 loss 은 개별 손실의 가중 합계가 됩니다.
model.evaluate(train_ds, return_dict=True)
361/361 [==============================] - 2s 4ms/step - loss: 0.3959 - duration_loss: 0.1213 - pitch_loss: 4.8476 - step_loss: 0.0322
{'loss': 0.39588069915771484,
'duration_loss': 0.12134315073490143,
'pitch_loss': 4.847629547119141,
'step_loss': 0.03215572610497475}
모델을 훈련시킵니다.
callbacks = [
tf.keras.callbacks.ModelCheckpoint(
filepath='./training_checkpoints/ckpt_{epoch}',
save_weights_only=True),
tf.keras.callbacks.EarlyStopping(
monitor='loss',
patience=5,
verbose=1,
restore_best_weights=True),
]
%%time
epochs = 50
history = model.fit(
train_ds,
epochs=epochs,
callbacks=callbacks,
)
Epoch 1/50 361/361 [==============================] - 4s 5ms/step - loss: 0.3075 - duration_loss: 0.0732 - pitch_loss: 4.0974 - step_loss: 0.0294 Epoch 2/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2950 - duration_loss: 0.0696 - pitch_loss: 3.9526 - step_loss: 0.0278 Epoch 3/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2927 - duration_loss: 0.0682 - pitch_loss: 3.9372 - step_loss: 0.0276 Epoch 4/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2918 - duration_loss: 0.0681 - pitch_loss: 3.9232 - step_loss: 0.0275 Epoch 5/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2874 - duration_loss: 0.0657 - pitch_loss: 3.9079 - step_loss: 0.0264 Epoch 6/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2842 - duration_loss: 0.0653 - pitch_loss: 3.8509 - step_loss: 0.0263 Epoch 7/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2820 - duration_loss: 0.0650 - pitch_loss: 3.8090 - step_loss: 0.0265 Epoch 8/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2806 - duration_loss: 0.0654 - pitch_loss: 3.7903 - step_loss: 0.0257 Epoch 9/50 361/361 [==============================] - 2s 6ms/step - loss: 0.2806 - duration_loss: 0.0651 - pitch_loss: 3.7888 - step_loss: 0.0261 Epoch 10/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2778 - duration_loss: 0.0637 - pitch_loss: 3.7690 - step_loss: 0.0256 Epoch 11/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2762 - duration_loss: 0.0624 - pitch_loss: 3.7704 - step_loss: 0.0253 Epoch 12/50 361/361 [==============================] - 2s 6ms/step - loss: 0.2746 - duration_loss: 0.0616 - pitch_loss: 3.7644 - step_loss: 0.0248 Epoch 13/50 361/361 [==============================] - 2s 6ms/step - loss: 0.2728 - duration_loss: 0.0604 - pitch_loss: 3.7591 - step_loss: 0.0244 Epoch 14/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2710 - duration_loss: 0.0584 - pitch_loss: 3.7573 - step_loss: 0.0247 Epoch 15/50 361/361 [==============================] - 2s 6ms/step - loss: 0.2694 - duration_loss: 0.0574 - pitch_loss: 3.7610 - step_loss: 0.0239 Epoch 16/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2686 - duration_loss: 0.0569 - pitch_loss: 3.7529 - step_loss: 0.0240 Epoch 17/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2695 - duration_loss: 0.0577 - pitch_loss: 3.7486 - step_loss: 0.0243 Epoch 18/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2663 - duration_loss: 0.0560 - pitch_loss: 3.7473 - step_loss: 0.0229 Epoch 19/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2642 - duration_loss: 0.0543 - pitch_loss: 3.7366 - step_loss: 0.0231 Epoch 20/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2691 - duration_loss: 0.0587 - pitch_loss: 3.7421 - step_loss: 0.0233 Epoch 21/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2636 - duration_loss: 0.0547 - pitch_loss: 3.7314 - step_loss: 0.0223 Epoch 22/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2613 - duration_loss: 0.0533 - pitch_loss: 3.7313 - step_loss: 0.0215 Epoch 23/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2595 - duration_loss: 0.0516 - pitch_loss: 3.7219 - step_loss: 0.0218 Epoch 24/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2548 - duration_loss: 0.0493 - pitch_loss: 3.7148 - step_loss: 0.0198 Epoch 25/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2539 - duration_loss: 0.0483 - pitch_loss: 3.7150 - step_loss: 0.0199 Epoch 26/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2526 - duration_loss: 0.0474 - pitch_loss: 3.7138 - step_loss: 0.0196 Epoch 27/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2502 - duration_loss: 0.0460 - pitch_loss: 3.7036 - step_loss: 0.0190 Epoch 28/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2467 - duration_loss: 0.0442 - pitch_loss: 3.6970 - step_loss: 0.0177 Epoch 29/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2458 - duration_loss: 0.0438 - pitch_loss: 3.6938 - step_loss: 0.0172 Epoch 30/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2434 - duration_loss: 0.0418 - pitch_loss: 3.6836 - step_loss: 0.0174 Epoch 31/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2404 - duration_loss: 0.0403 - pitch_loss: 3.6703 - step_loss: 0.0166 Epoch 32/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2421 - duration_loss: 0.0412 - pitch_loss: 3.6833 - step_loss: 0.0168 Epoch 33/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2391 - duration_loss: 0.0399 - pitch_loss: 3.6585 - step_loss: 0.0163 Epoch 34/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2376 - duration_loss: 0.0390 - pitch_loss: 3.6467 - step_loss: 0.0163 Epoch 35/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2403 - duration_loss: 0.0417 - pitch_loss: 3.6448 - step_loss: 0.0164 Epoch 36/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2394 - duration_loss: 0.0417 - pitch_loss: 3.6218 - step_loss: 0.0166 Epoch 37/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2337 - duration_loss: 0.0369 - pitch_loss: 3.6155 - step_loss: 0.0161 Epoch 38/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2320 - duration_loss: 0.0357 - pitch_loss: 3.6080 - step_loss: 0.0158 Epoch 39/50 361/361 [==============================] - 2s 6ms/step - loss: 0.2291 - duration_loss: 0.0353 - pitch_loss: 3.5896 - step_loss: 0.0143 Epoch 40/50 361/361 [==============================] - 2s 6ms/step - loss: 0.2285 - duration_loss: 0.0352 - pitch_loss: 3.5784 - step_loss: 0.0144 Epoch 41/50 361/361 [==============================] - 2s 6ms/step - loss: 0.2276 - duration_loss: 0.0338 - pitch_loss: 3.5928 - step_loss: 0.0142 Epoch 42/50 361/361 [==============================] - 2s 6ms/step - loss: 0.2233 - duration_loss: 0.0316 - pitch_loss: 3.5582 - step_loss: 0.0137 Epoch 43/50 361/361 [==============================] - 2s 6ms/step - loss: 0.2211 - duration_loss: 0.0304 - pitch_loss: 3.5453 - step_loss: 0.0134 Epoch 44/50 361/361 [==============================] - 2s 6ms/step - loss: 0.2206 - duration_loss: 0.0307 - pitch_loss: 3.5396 - step_loss: 0.0129 Epoch 45/50 361/361 [==============================] - 2s 6ms/step - loss: 0.2223 - duration_loss: 0.0322 - pitch_loss: 3.5352 - step_loss: 0.0133 Epoch 46/50 361/361 [==============================] - 2s 6ms/step - loss: 0.2213 - duration_loss: 0.0312 - pitch_loss: 3.5323 - step_loss: 0.0135 Epoch 47/50 361/361 [==============================] - 2s 6ms/step - loss: 0.2240 - duration_loss: 0.0329 - pitch_loss: 3.5405 - step_loss: 0.0142 Epoch 48/50 361/361 [==============================] - 2s 6ms/step - loss: 0.2217 - duration_loss: 0.0322 - pitch_loss: 3.5160 - step_loss: 0.0137 Epoch 49/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2167 - duration_loss: 0.0296 - pitch_loss: 3.4894 - step_loss: 0.0126 Epoch 50/50 361/361 [==============================] - 2s 5ms/step - loss: 0.2142 - duration_loss: 0.0278 - pitch_loss: 3.4757 - step_loss: 0.0126 CPU times: user 2min 16s, sys: 23.9 s, total: 2min 40s Wall time: 1min 41s
plt.plot(history.epoch, history.history['loss'], label='total loss')
plt.show()
메모 생성
모델을 사용하여 메모를 생성하려면 먼저 메모의 시작 순서를 제공해야 합니다. 아래 함수는 일련의 음표에서 하나의 음표를 생성합니다.
음높이의 경우 모델에서 생성된 음표의 softmax 분포에서 샘플을 추출하며 단순히 가장 높은 확률의 음표를 선택하지 않습니다. 항상 확률이 가장 높은 음표를 선택하면 음표가 반복적으로 생성됩니다.
temperature 매개변수는 생성된 음표의 무작위성을 제어하는 데 사용할 수 있습니다. RNN을 사용한 텍스트 생성 에서 온도에 대한 자세한 내용을 찾을 수 있습니다.
def predict_next_note(
notes: np.ndarray,
keras_model: tf.keras.Model,
temperature: float = 1.0) -> int:
"""Generates a note IDs using a trained sequence model."""
assert temperature > 0
# Add batch dimension
inputs = tf.expand_dims(notes, 0)
predictions = model.predict(inputs)
pitch_logits = predictions['pitch']
step = predictions['step']
duration = predictions['duration']
pitch_logits /= temperature
pitch = tf.random.categorical(pitch_logits, num_samples=1)
pitch = tf.squeeze(pitch, axis=-1)
duration = tf.squeeze(duration, axis=-1)
step = tf.squeeze(step, axis=-1)
# `step` and `duration` values should be non-negative
step = tf.maximum(0, step)
duration = tf.maximum(0, duration)
return int(pitch), float(step), float(duration)
이제 몇 가지 메모를 생성합니다. next_notes 에서 온도와 시작 순서를 가지고 놀고 무슨 일이 일어나는지 볼 수 있습니다.
temperature = 2.0
num_predictions = 120
sample_notes = np.stack([raw_notes[key] for key in key_order], axis=1)
# The initial sequence of notes; pitch is normalized similar to training
# sequences
input_notes = (
sample_notes[:seq_length] / np.array([vocab_size, 1, 1]))
generated_notes = []
prev_start = 0
for _ in range(num_predictions):
pitch, step, duration = predict_next_note(input_notes, model, temperature)
start = prev_start + step
end = start + duration
input_note = (pitch, step, duration)
generated_notes.append((*input_note, start, end))
input_notes = np.delete(input_notes, 0, axis=0)
input_notes = np.append(input_notes, np.expand_dims(input_note, 0), axis=0)
prev_start = start
generated_notes = pd.DataFrame(
generated_notes, columns=(*key_order, 'start', 'end'))
generated_notes.head(10)
out_file = 'output.mid'
out_pm = notes_to_midi(
generated_notes, out_file=out_file, instrument_name=instrument_name)
display_audio(out_pm)
아래 두 줄을 추가하여 오디오 파일을 다운로드할 수도 있습니다.
from google.colab import files
files.download(out_file)
생성된 메모를 시각화합니다.
plot_piano_roll(generated_notes)
pitch , step , duration 의 분포 를 확인 하십시오 .
plot_distributions(generated_notes)
위의 플롯에서 메모 변수의 분포가 변경되었음을 알 수 있습니다. 모델의 출력과 입력 사이에 피드백 루프가 있기 때문에 모델은 손실을 줄이기 위해 유사한 출력 시퀀스를 생성하는 경향이 있습니다. 이것은 특히 MSE 손실을 사용하는 step 및 duration 과 관련이 있습니다. pitch 의 경우 predict_next_note 에서 temperature 를 높여 임의성을 높일 수 있습니다.
다음 단계
이 튜토리얼은 RNN을 사용하여 MIDI 파일 데이터 세트에서 일련의 음표를 생성하는 방법을 보여주었습니다. 더 자세히 알아보려면 추가 다이어그램과 설명이 포함 된 RNN 자습서로 밀접하게 관련된 텍스트 생성을 방문하세요.
음악 생성을 위해 RNN을 사용하는 것의 대안은 GAN을 사용하는 것입니다. 오디오를 생성하는 대신 GAN 기반 접근 방식은 전체 시퀀스를 병렬로 생성할 수 있습니다. Magenta 팀은 GANSynth 를 사용하여 이 접근 방식에 대해 인상적인 작업을 수행했습니다. Magenta 프로젝트 웹사이트 에서 멋진 음악 및 예술 프로젝트와 오픈 소스 코드를 찾을 수도 있습니다.