[Original date of post creation: 2024-06-02]
Hello everyone!
For a long time, I wanted to write a guide with my process of donwloading and tagging songs (I can use it as a guide for me in case I forget it, haha). Anything here is a suggestion, you can do it differently if you want. (This guide is for windows (10) devices, linux tools may be different. I haven't moved to Linux, yet..) This is gonna be long.
Tools we may need:
(online means that they don't have to be installed)
-- Deemix to download songs from deezer (open source).
-- Soulseek platfrom to share music. There is an android open source client SeekerAndroid and a PC open source client Nicotine+.
-- TuneMyMusic website to create/edit/import/export playlists to/from/in various music platfrom (online). It can also export in text.
-- ytDownloader to download songs from Youtube (for PC, open source). It can also download videos/songs from many other sources, acts as a gui for yt-dpl.
-- Seal to download songs from Youtube (for android, open source). It can also download videos/songs from many other sources, acts as a gui for yt-dpl.
-- Mp3tag (for Linux you can use puddletag, but I can't help you with that as of now) to manually (and batch) edit tags of my downloaded songs (closed source freeware, until recently I thought it was open source.. I might replace it with Kid3 if I ever get to know how to use it). Mp3tag has "actions" which help you do batch edit of metadata, like format the filename of a song using the artist and title tags (and with ctrl+a you can select all songs to apply that action).
-- OneTagger to automatically tag songs (open source). It can fetch metadata from various platforms, like musicbrainz, deeer, spotify, musixmatch, bandcamp, discogs etc.
-- LyricsGenius python program to fetch data from genius using genius api (open source).
-- lrcget to batch fetch lyrics from lrclib (open source).
-- lrclib website with synced and unsynced lyrics (online).
-- Genius website with unsynced lyrics, song metadata and lyrics interpretations (online).
-- Notepad++ to edit text (open source).
-- fre:ac to convert my files from .flac to .opus (open source). It can do many more, it can also rip CDs etc.
-- Quodlibet windows/linux/macos music player (open source). Very good.
-- Symphony android music player (open source). Very good.
-- ListenBrainz to record my listening history (open source, online). It has an offcial open source android client too. Many music players (on pc at least) have plug-ins that support ListenBrainz.
-- Noteable mentions: Simplest file renamer to batch rename files or get a list of the filenames in a folder (open source) and Libre Office to use the Office calc funtions (open source).
The download part:
I will be using Deemix as my primary source, Soulseek for anything else and Youtube when I can't find my song anywhere else.
Deemix needs an account, so you can probably find a ARL here (arl is like a token for an account). If the link doesn't work in the future, you'll have to find it yourself. Soulseek simply needs a free account (but there it is encouraged to share your music with others).
In Deemix you can search songs seperately or download playlists (either from deemix or spotify). You can create spotify playlists from various sources (even text list) with TuneMyMusic.
Deemix settings:
-- Use this trackname template in all occasions:
%artist% - %title% [%year%]
-- Save covers at 1200 size in .png format
-- Save all tags apart from copyright
-- Set concurent downloads to 1, because it makes sure songs from playlists get downloaded with the same order in the playlist (it helps me check later what has been downloaded properly) and makes it harder to be detected.
-- Set it to download Flac and set it to create .lrc files
-- Turn on Keep "Various artists" in the Album artists
-- Use YYYY-MM-DD date format
-- Set it to remove "featured artists" from the title
-- Add your Spotify account with the proper tokens/credentials
Youtube: either use yt-dpl or ytDonloader (it has a gui)
The tagging part (the longest part):
Our main tools will be Mp3tag and OneTagger. OneTagger will fetch the tags and Mp3tag will edit them.
First, we will create our folder structure which we will use:
-- Create a folder 1T.
-- Inside create 10 more: 0 Raw, 1 Deezer, 2 Spotify, 3 Musicbrainz, 4 Bandcamp, 5 Musixmatch, 6 Opus, 7 Final, Images, Texts.
-- Inside the 7 Final folder create 2 more: Flac and Opus.
I will first describe how to set up OneTagger, Mp3tag and anything else and then I'll describe the steps.
OneTagger settings:
(!Attention!) I mention any changes from the default, anything else I'll probably not mention it.
Some terms explained before we start:
-- ReleaseID is the part after the last / of the deezer url of the album which the song belongs to. i.e. from this link https://www.deezer.com/en/album/5447741 the id is 5447741.
-- TrackID is the part after the last / of the deezer url of the album which the song belongs to. If you download a song from Deemix, it usually has the TrackID saved in the SOURCEID tag. i.e. from this link https://www.deezer.com/en/track/55848051 the id is 55848051.
First we will create our AutoTag profiles:
1) Create new profile 1 Deezer
(This is for songs downloaded with Deemix, the 6th profile is the universal alternative.)
-- In platforms tab select Deezer as platform
-- In tags tab, select Version, Label, ReleaseID, Catalog Number, Disc Number, OneTagger Tags. (Select album art only if you want to change the art or you haven't downloaded the songs from deemix). Select as input the path of the folder 0 Raw.
-- In platform specific settings set Album Art resolution to 1600 (Redundant step)
-- In advanced turn on only ID3v2.4, Identify tracks with Shazam, Move successfully tagged files after taggins and add the path of the 1 Deezer folder underneath. In the overwrite tags area, add those tags: Version, Label, ReleaseID, Catalog Number, Disc Number, OneTagger Tags. Set track number leading zeros to 2 and use // as separators.
2) Create new profile 2 Spotify
-- In platforms tab select Spotify as platform
-- In tags tab, select Album Artist, ReleaseID, TrackID, URL. Select as input the path of the folder 1 Deezer.
-- In platform specific settings put in your Spotify credentials
-- In advanced turn on only ID3v2.4, Identify tracks with Shazam, Move successfully tagged files after taggins and add the path of the 2 Spotify folder underneath. In the overwrite tags area, add those tags: Album Artist, ReleaseID, TrackID, URL. Set track number leading zeros to 2 and use // as separators.
3) Create new profile 3 Musicbrainz
-- In platforms tab select Musicbrainz as platform
-- In tags tab, select Genre, ReleaseID, TrackID, URL. Select as input the path of the folder 2 Spotify.
-- In advanced turn on only ID3v2.4, Identify tracks with Shazam, Move successfully tagged files after taggins and add the path of the 3 Musicbrainz folder underneath. In the overwrite tags area, add those tags: Genre, ReleaseID, TrackID, URL. Set track number leading zeros to 2 and use // as separators.
4) Create new profile 4 Bandcamp
-- In platforms tab select Bandcamp as platform
-- In tags tab, select Genre, Style/Subgenre, ReleaseID, TrackID. Select as input the path of the folder 3 Musicbrainz.
-- In advanced turn on only ID3v2.4, Identify tracks with Shazam, Move successfully tagged files after taggins and add the path of the 4 Bandcamp folder underneath. In the overwrite tags area, add those tags: Genre, Style/Subgenre, ReleaseID, TrackID. Set track number leading zeros to 2 and use // as separators. In Genre/Style/Subgenre options select Merge to Genre tag.
5) Create new profile 5 Musixmatch
-- In platforms tab select Musixmatch as platform
-- In tags tab, select Unsynced Lyrics, Synced Lyrics. Select as input the path of the folder 4 Bandcamp.
-- In advanced turn on only ID3v2.4, Identify tracks with Shazam, Write .LRC file, Move successfully tagged files after taggins and add the path of the 5 Musixmatch folder underneath. In the overwrite tags area, add those tags: Unsynced Lyrics, Synced Lyrics. Set track number leading zeros to 2 and use // as separators. In Genre/Style/Subgenre options select Merge to Genre tag.
Extra/Secondary profiles to use when you don't tag songs downloaded from Deemix:
6) Create new profile 1 Deezer - Universal
-- In platforms tab select Deezer as platform
-- In tags tab, select everything apart from Explicit and Album Art (select album art only if you want to change the art or you haven't downloaded the songs from deemix). Select as input the path of the folder 0 Raw.
-- In platform specific settings set Album Art resolution to 1600 (Redundant step)
-- In advanced turn on only Overwrite tags, ID3v2.4, Identify tracks with Shazam, Move successfully tagged files after taggins and add the path of the 1 Deezer folder underneath. Set track number leading zeros to 2 and use // as separators.
7) Create new profile 2 Spotify - Universal
-- In platforms tab select Spotify as platform
-- In tags tab, select Artist, Album Artist, ReleaseID, TrackID, ISRC, URL. Select as input the path of the folder 1 Deezer.
-- In platform specific settings put in your Spotify credentials
-- In advanced turn on only ID3v2.4, Identify tracks with Shazam, Move successfully tagged files after taggins and add the path of the 2 Spotify folder underneath. In the overwrite tags area, add those tags: Artist, ReleaseID, TrackID, ISRC, URL. Set track number leading zeros to 2 and use // as separators.
8) Create new profile 3 Musicbrainz - Universal
-- In platforms tab select Musicbrainz as platform
-- In tags tab, select Genre, ReleaseID, TrackID, ISRC, URL. Select as input the path of the folder 2 Spotify.
-- In advanced turn on only ID3v2.4, Identify tracks with Shazam, Move successfully tagged files after taggins and add the path of the 3 Musicbrainz folder underneath. In the overwrite tags area, add those tags: Genre, ReleaseID, TrackID, ISRC, URL. Set track number leading zeros to 2 and use // as separators.
These are all the profiles we will need.
Next we will set up the Audio Features tab:
-- Put in your spotify credentials and log in.
-- In the Select Input field put the path of the 5 Musixmatch folder.
-- Leave properties as they are.
-- In separators section use // as the separators.
-- In advanced section enable the Write OneTagger meta tag and Include subfolders. Disable the Skip already tagged tracks.
And now OneTagger is set up.
Mp3tag setup:
1) First we will create an export configuration which we will use later:
-- Create a new configuration csv for py script (only artist and title), and edit it using this as the code:
$filename(titleartistlist.csv,utf-16)artist title
$loop(%_filename_ext%)$meta_sep(artist,$char(44) ) %title%
$loopend()
-- Set export filename to titleartistlist.csv and add behind it the path of the Texts folder we created before (we want the exported file to be inside the Texts folder)
2) Then we will create our actions:
(I will write the name and the steps needed to create each action)
1. wwwaudio to wwwaudiosource (Deezer) from Deemix
-- Create a format action type: set field to WWWAUDIOSOURCE and format string to https://www.deezer.com/track/%sourceid%
2. wwwaudio to www (Spotify)
-- Create a format action type: set field to WWW and format string to %wwwaudiofile%
3a. Format 1 rating
-- Create a format action type: set field to RATING and format string to 20
-- Create a format action type: set field to RATING WMP and format string to 1
3b. Format 1 rating
-- Create a format action type: set field to RATING and format string to 40
-- Create a format action type: set field to RATING WMP and format string to 64
3c. Format 1 rating
-- Create a format action type: set field to RATING and format string to 60
-- Create a format action type: set field to RATING WMP and format string to 128
3d. Format 1 rating
-- Create a format action type: set field to RATING and format string to 80
-- Create a format action type: set field to RATING WMP and format string to 192
3e. Format 1 rating
-- Create a format action type: set field to RATING and format string to 100
-- Create a format action type: set field to RATING WMP and format string to 255
4. Filename format artist - title [year]( -- [YT])
-- Create a format action type: set field to _FILENAME and format string to $meta_sep(artist,; ) - %title% $char(91)$left(%year%,4)$char(93)$if($geql($strstr($meta_sep(genre,\\),$char(91)YT$char(93)),1), -- $char(91)YT$char(93),)
-- Note: $char(91) is [ and $char(93) is ]
4b. Filename format artist - title [year]( -- [YT])
-- Create a format action type: set field to _FILENAME and format string to $meta_sep(artist,; ) - %title%$if($geql($strstr($meta_sep(genre,\\),$char(91)YT$char(93)),1), -- $char(91)YT$char(93),)
5. Add-Fix [Xstar(s)] genre
-- Create a format action type: set field to GENRE and format string to $if($or($geql($strstr($meta_sep(genre,\\),$char(91)5stars$char(93)),1),$geql($strstr($meta_sep(genre,\\),$char(91)4stars$char(93)),1),$geql($strstr($meta_sep(genre,\\),$char(91)3stars$char(93)),1),$geql($strstr($meta_sep(genre,\\),$char(91)2stars$char(93)),1),$geql($strstr($meta_sep(genre,\\),$char(91)1star$char(93)),1),$geql($strstr($meta_sep(genre,\\),$char(91)0star$char(93)),1)),$replace($meta_sep(genre,\\\\),$char(91)5stars$char(93),$char(91)$div(%rating%,20)star$ifgreater(%rating%,20,s,)$char(93),$char(91)4stars$char(93),$char(91)$div(%rating%,20)star$ifgreater(%rating%,20,s,)$char(93),$char(91)3stars$char(93),$char(91)$div(%rating%,20)star$ifgreater(%rating%,20,s,)$char(93),$char(91)2stars$char(93),$char(91)$div(%rating%,20)star$ifgreater(%rating%,20,s,)$char(93),$char(91)1star$char(93),$char(91)$div(%rating%,20)star$ifgreater(%rating%,20,s,)$char(93),$char(91)0star$char(93),$char(91)$div(%rating%,20)star$ifgreater(%rating%,20,s,)$char(93)),$meta_sep(genre,\\\\)\\\\$char(91)$div(%rating%,20)star$ifgreater(%rating%,20,s,)$char(93))
6. Replace hiphoprap and Capitalise Genre
-- Create a replace action type: set field to GENRE, original to Hip-hop-rap and replace with Rap/Hip-Hop
-- Create a replace action type: set field to GENRE, original to Rap/Hip Hop and replace with Rap/Hip-Hop
-- Create a case conversion action type: set field to GENRE, case conversion to Mixed Case and words begin from/after any of ;/&[- (the spacebar is not a typo)
7. Synced Lyrics from lrc
(this has a lot of steps because we use regular expressions to fix the line breaks)
-- Create an Import text file action type: set field to SYNCEDLYRICS and filename %_filename%.lrc
-- Create a replace with regular expression action type: set field to SYNCEDLYRICS, regular expression to \]\x20, replace with \] and don't use case-sensitive comparison
-- Create a replace with regular expression action type: set field to SYNCEDLYRICS, regular expression to \]\x20, replace with \] and don't use case-sensitive comparison
-- Create a replace with regular expression action type: set field to SYNCEDLYRICS, regular expression to (\]\r\n|\]\n), replace with \]\x{266A}\n and don't use case-sensitive comparison
-- Create a replace with regular expression action type: set field to SYNCEDLYRICS, regular expression to \r, replace with `` (nothing, blank) and don't use case-sensitive comparison
-- Create a format action type: set field to SYNCEDLYRICS and format string to $if($strcmp($right(%syncedlyrics%,1),$char(93)),%syncedlyrics%$char(9834),%syncedlyrics%)
-- Create a format action type: set field to SYNCEDLYRICS and format string to $char(91)ti:%title%$char(93)$char(10)$char(91)ar:%artist%$char(93)$char(10)%syncedlyrics%
-- Create a format action type: set field to LYRICS and format string to %syncedlyrics%
-- Create a replace with regular expression action type: set field to LYRICS, regular expression to (\[\d\d\:\d\d.\d\d\]|\[\d\d\:\d\d.\d\d\d]), replace with `` (nothing, blank) and don't use case-sensitive comparison
-- Create a format action type: set field to UNSYNCEDLYRICS and format string to %lyrics%
8. Export artist and title for genius and backup year and cover
-- Create an export action type: pick the csv for py script (only artist and title) we've created and set export filename to titleartistlist.csv and add behind it the path of the Texts folder we created before (we want the exported file to be inside the Texts folder)
-- Create a format action type: set field to DEEZER_YEAR and format string to %year%
-- Create an export cover to file action and in the format string add the path to the Images\Old\%_filename%_Deezer (from the Images folder we've created). This will act as a backup of our images. Note that it adds the _Deezer name at the end of each filename, this may cause inaccuracies if we use songs that have images from different sources.
9. Merge, trim, and genius Year
-- Create a merge duplicate fields action type: set field to YEAR and separator to \\
-- Create a replace action type: set field to GENIUS_YEAR, original to None and replace with 00
-- Create a replace with regular expression action type: set field to GENIUS_YEAR, regular expression to (\d+)-(\d+)-(\d+), replace with $1-$if($eql($len($2),1),0$2,$2)-$if($eql($len($3),1),0$3,$3) and don't use case-sensitive comparison
-- Create a format action type: set field to YEAR and format string to $if($grtr($len(%year%),10),$cutLeft(%year%,6),%year%)
-- Create a format action type: set field to YEAR and format string to $if($or($eql($left(%genius_year%,1),0),$and($strstr(%genius_year%,$char(45)00$char(45)00),$and($eql($len(%deezer_year%),10),$leql($if($less($sub($left(%genius_year%,4),$left(%deezer_year%,4)),0),$mul(-1,$sub($left(%genius_year%,4),$left(%deezer_year%,4))),$sub($left(%genius_year%,4),$left(%deezer_year%,4))),1)))),%deezer_year%,%genius_year%)
Genre adding [YT]
-- Create a format action type: set field to GENRE and format string to $if($eql($strstr($meta_sep(genre,\\),$char(91)YT$char(93)),0),$meta_sep(genre,\\\\)\\\\$char(91)YT$char(93),$meta_sep(genre,\\\\))
Genre adding [Instrument@l]
-- Create a format action type: set field to GENRE and format string to $if($eql($strstr($meta_sep(genre,\\),$char(91)Instrument$char(64)l$char(93)),0),$meta_sep(genre,\\\\)\\\\$char(91)Instrument$char(64)l$char(93),$meta_sep(genre,\\\\))
Genre adding [Gr33k]
-- Create a format action type: set field to GENRE and format string to $if($eql($strstr($meta_sep(genre,\\),$char(91)Gr33k$char(93)),0),$meta_sep(genre,\\\\)\\\\$char(91)Gr33k$char(93),$meta_sep(genre,\\\\))
Genre adding [8b@ll]
-- Create a format action type: set field to GENRE and format string to $if($eql($strstr($meta_sep(genre,\\),$char(91)8B$char(64)ll$char(93)),0),$meta_sep(genre,\\\\)\\\\$char(91)8B$char(64)ll$char(93),$meta_sep(genre,\\\\))
Genre adding [Nitr0]
-- Create a format action type: set field to GENRE and format string to $if($eql($strstr($meta_sep(genre,\\),$char(91)Nitr0$char(93)),0),$meta_sep(genre,\\\\)\\\\$char(91)Nitr0$char(93),$meta_sep(genre,\\\\))
Genre adding [F@vourites]
-- Create a format action type: set field to GENRE and format string to $if($eql($strstr($meta_sep(genre,\\),$char(91)F$char(64)vourites$char(93)),0),$meta_sep(genre,\\\\)\\\\$char(91)F$char(64)vourites$char(93),$meta_sep(genre,\\\\))
Remove favourite genre
-- Create a format action type: set field to GENRE and format string to $if($eql($strstr($meta_sep(genre,\\),$char(91)F$char(64)vourites$char(93)),0),$meta_sep(genre,\\\\),$replace($meta_sep(genre,\\\\),$char(91)F$char(64)vourites$char(93),))
Source to Deezer
-- Create a format action type: set field to SOURCE and format string to Deezer
wwwaudio to wwwaudiosource (Deezer)
-- Create a format action type: set field to WWWAUDIOSOURCE and format string to %wwwaudiofile%
Merge and split tags
-- Create a merge duplicate fields action type: set field to ALBUMARTIST and separator to \\
-- Create a replace action type: set field to ALBUMARTIST, original to \\ and replace with ;
-- Create a merge duplicate fields action type: set field to ARTIST and separator to \\
-- Create a replace action type: set field to ARTIST, original to \\ and replace with ;
-- Create a merge duplicate fields action type: set field to GENRE and separator to \\
-- Create a replace action type: set field to GENRE, original to \\ and replace with ;
-- Create a merge duplicate fields action type: set field to PUBLISHER and separator to \\
-- Create a replace action type: set field to PUBLISHER, original to \\ and replace with ;
-- Create a merge duplicate fields action type: set field to AUTHOR and separator to \\
-- Create a replace action type: set field to AUTHOR, original to \\ and replace with ;
-- Create a merge duplicate fields action type: set field to COMPOSER and separator to \\
-- Create a replace action type: set field to COMPOSER, original to \\ and replace with ;
-- Create a merge duplicate fields action type: set field to AUDIO_FEATURES and separator to \\
-- Create a replace action type: set field to AUDIO_FEATURES, original to \\ and replace with ;
-- Create a merge duplicate fields action type: set field to LABEL and separator to \\
-- Create a replace action type: set field to LABEL, original to \\ and replace with ;
; to double inverted slash
-- Create a replace action type: set field to ARTIST, original to ; and replace with \\
-- Create a replace action type: set field to GENRE, original to ; and replace with \\
-- Create a replace action type: set field to ALBUMARTIST, original to ; and replace with \\
-- Create a replace action type: set field to AUDIO_FEATURES, original to ; and replace with \\
-- Create a replace action type: set field to PUBLISHER, original to ; and replace with \\
-- Create a replace action type: set field to LABEL, original to ; and replace with \\
-- Create a replace action type: set field to COMPOSER, original to ; and replace with \\
-- Create a replace action type: set field to AUTHOR, original to ; and replace with \\
And these were all the actions we needed!
3) Next we will create our python script that will fetch data from genius:
For this we first need to install the LyricsGenius program from github. Our script will use that program.
I'll just copy paste the code. Name it genius_data_fetcher.py and saved it in the Texts folder we've created.
This script takes a titleartistlist.csv list we create and it creates a geniusdata.txt file which contains the following items separated by tab: geniusID, url, genius song cover url, genius song cover url, arists and title. It also adds in the begging the date or at least the year or year-month (if they exist, otherwise their data slots are filled with zeroes). Lastly it adds in the end of each line the artists and the title of each song it reads from the list. Currently I have disabled/commented out the part that downloads the images, as it's not much necessary.
Remeber to change the path to match the appropriate folders on your device and use your own Genius token (replace the PlaceholderForYourGeniusToken)!
code
import csv
import requests
from lyricsgenius import Genius
genius = Genius('PlaceholderForYourGeniusToken', timeout=20, sleep_time=1)
'''
#imports two .txt
#importing title list
with open("title_list.txt",'r', encoding='utf-16') as titles_file:
titlelist = titles_file.read().splitlines()
titles_file.close()
# importing artist list
with open("artist_list.txt",'r', encoding='utf-16') as artist_file:
artistlist = artist_file.read().splitlines()
artist_file.close()
'''
titlelist = []
artistlist = []
with open('C:\\Users\BS\\Downloads\\1T\\Texts\\titleartistlist.csv', mode='r', encoding='utf-16') as csv_file:
csv_reader = csv.DictReader(csv_file, delimiter='\t')
#line_count = 0
for row in csv_reader:
'''if line_count == 0:
#print(f'Column names are {", ".join(row)}')
#print(row["title"])
line_count += 1
#print(f'{row["title"]} ; {row["title"]}')
else:'''
#print(row["artist"])
artistlist.append(row["artist"])
titlelist.append(row["title"])
#line_count += 1
#print(f'Processed {line_count} lines.\n')
array_length = len(artistlist)
#searching genius
with open('C:\\Users\BS\\Downloads\\1T\\Texts\\geniusdata.txt', 'w', encoding='utf-16') as gd:
for i in range(array_length):
SArtist = artistlist[i]
STitle = titlelist[i]
song = genius.search_song(STitle, SArtist)
if song is None:
gd.write('0000-00-00\t0000\t0000\t0000\t0000\t0000') #00 or 0000 as error code
gd.write('\t')
gd.write(SArtist)
gd.write('\t')
gd.write(STitle)
gd.write('\n')
else:
song_dict = song.to_dict()
RDate=song_dict.get('release_date_components')
if RDate is None:
line = ['0000-00-00\t',
str(song_dict.get('id')),'\t',
song_dict.get('url'),'\t',
song_dict.get('song_art_image_url'),'\t',
song_dict.get('artist_names'),'\t',
song_dict.get('title'),'\t',
SArtist,'\t',STitle,'\n']
gd.writelines(line)
#uncomment the next two blocks of code if you want to download the images as well
'''image_url = song_dict.get('song_art_image_url') #downloads image
default_url_check = image_url.find('default_cover_image.png') #checks default image
print(default_url_check)
if default_url_check == -1:
file_destination = ''.join(['C:\\Users\BS\\Downloads\\1T\\Images\\Genius\\',image_url.rsplit('/', 1)[-1]])
res = requests.get(image_url)
if res.status_code == 200: # http 200 means success
with open(file_destination, 'wb') as file_handle: # wb means Write Binary
file_handle.write(res.content)
file_handle.close()
print('Image downloaded')
else:
print('Default image, skip.')'''
else:
line = [str(RDate.get('year')),'-',
str(RDate.get('month')),'-',
str(RDate.get('day')),'\t',
str(song_dict.get('id')),'\t',
song_dict.get('url'),'\t',
song_dict.get('song_art_image_url'),'\t',
song_dict.get('artist_names'),'\t',
song_dict.get('title'),'\t',
SArtist,'\t',STitle,'\n']
'''image_url = song_dict.get('song_art_image_url') #downloads image
default_url_check = image_url.find('default_cover_image') #checks default image
if default_url_check == -1:
file_destination = ''.join(['C:\\Users\BS\\Downloads\\1T\\Images\\Genius\\',image_url.rsplit('/', 1)[-1]])
res = requests.get(image_url)
if res.status_code == 200: # http 200 means success
with open(file_destination, 'wb') as file_handle: # wb means Write Binary
file_handle.write(res.content)
file_handle.close()
print('Image downloaded')
else:
print('Default image, skip.')'''
gd.writelines(line)
gd.close()
print('Finished!!\nGo to:\nC:\\Users\BS\\Downloads\\1T\\Texts\\')
#['_type', 'annotation_count', 'api_path', 'artist_names', 'full_title', 'header_image_thumbnail_url', 'header_image_url', 'id', 'instrumental', 'lyrics_owner_id', 'lyrics_state', 'lyrics_updated_at', 'path', 'pyongs_count', 'relationships_index_url', 'release_date_components', 'release_date_for_display', 'release_date_with_abbreviated_month_for_display', 'song_art_image_thumbnail_url', 'song_art_image_url', 'stats', 'title', 'title_with_featured', 'updated_by_human_at', 'url', 'featured_artists', 'primary_artist', 'artist', 'lyrics']
4) Now that we have our python script, we will create a batch script that will call the python script:
-- Name it GeniusFetcherCall.bat and save it inside the Texts folder. It's just 3 lines of code:
cd C:\Users\BS\Downloads\1T\Texts
python genius_data_fetcher.py
PAUSE
5) We will use this batch script to create a tool in Mp3tag:
(It makes it easier to call it without searching for the actual file.)
-- Go to File->Options->Tools->New
-- In the name field put Genius Data Fetcher and in the path field put the path of your batch script.
We have finished setting up Mp3tag!
Now it's time to actually tag the files:
-
Throw all your downloaded song files in 0 Raw folder. In case you also downloaded .lrc files, throw the .lrc files in the 5 Musixmatch folder.
-
Open OneTagger and run the 1. Deezer profile. (If the downloaded files aren't downloaded using Deemix, run the 1. Deezer - Universal profile instead.)
(!Attention!) Re-run if there are fails/skips. If there are fails again, re-run by enabling the Force Shazam option in the advanced tab (shazam seems to stop working after 20 (successfull?) scans, so you'll have to manually cut and paste the failed scaned songs to the 1 Deezer folder. Do this section every time we run a profile and you get failed/skipped songs. Remember to move the files to the appropriate folder. (Here we move them from 0 Raw to 1 Deezer manually, because we have the set profile to only move the successfully tagged files to the next folder. I won't repeat this section in the next steps to avoid repeating myself.
Once you're done with scanning, move all songs to the 1 Deezer folder.
Side note: failed songs probably won't have the Deezer ReleaseID and TrackID. You can add them manually (painfull) with the edit tag tab of OneTagger (or with Mp3tag), by searching online Deezer for each song separately.
-
Open Mp3tag.
(!Attention!) You must select the appropriate folder each time you use Mp3tag. This time we must open the 1 Deezer folder by pasting the path of the folder on the directory field located at the middle-bottom right of the window. We want to edit the newly tagged files each time, so we will change that directory after we focus on a different folder. I think simply opening the folder to where we last moved our files should be good. I want mention every time what Directory we will open.
Select all the songs and run the action 1. wwwaudio to wwwaudiosource (Deezer) from Deemix. If you didn't get the songs using Deemix, then choose wwwaudio to wwwaudiosource (Deezer) instead. This will copy the deezer url of the songs in the WWWAUDIOSOURCE tag.
-
In OneTagger run the 2. Spotify profile. (If the downloaded files aren't downloaded using Deemix, run the 2. Spotify - Universal profile instead.)
Once you're done with scanning, move all songs to the 3 Musicbrainz folder.
-
Open Mp3tag, select all the songs and run the action 2. wwwaudio to www (Spotify). This will copy the spotify url of the songs in the WWW tag.
-
In OneTagger run the 3. Musicbrainz profile. (If the downloaded files aren't downloaded using Deemix, run the 3. Musicbrainz - Universal profile instead.)
Once you're done with scanning, move all songs to the 4 Bandcamp folder.
-
In OneTagger run the 4. Bandcamp profile.
Once you're done with scanning, move all songs to the 5 Musixmatch folder.
-
In OneTagger run the 5. Musixmatch profile.
Once you're done with scanning, move the .lrc files (stored in 4 Bandcamp folder) to the 5 Musixmatch folder.
-
Open Mp3tag and tag/rate any songs you want using actions from 3a. to 3e. Each of those actions writes on two different tags, one for windows media player and one for the general rating tag. As you can see the 1/2/3/4/5 star scale corresponts to 1/64/128/192/255 in wmp rating and to 20/40/60/80/100 in rating.
-
In Mp3tag select all the songs and run the 4. Filename format artist - title [year]( -- [YT]) action. This will change the filename of all the songs by using the following template: Artist1; Artist2 - Title [Year] (not the date)
Each artist is separated by ; and the action adds the -- [YT] suffix only if we have added [YT] as genre in the song, which denotes that we got the song from Youtube.
The 4b. action is used just in case we don't want to add the [Year] suffix (yet).
-
In Mp3tag select all the songs and run the 5. Add-Fix [Xstar(s)] genre action. This action reads the RATING tag and adds the appropriate star rating in the GENRE tag. i.e. A song with rating 100 gets [5stars] in the genre tag. A song with 20 rating gets [1star].
-
In Mp3tag select all songs and run the 6. Replace hiphoprap and Capitalise Genre action. This fixes the Rap/Hip-Hop genre (some sources use different name for the same genre, so I replace them to be uniform) and capitalises the 1st letter of each word.
-
Now it's time to deal with lyrics. Musixmatch may not find all the lyrics or it may not find synced lyrics (on very rare occassions it may create black .lrc files??). Adding lyrics can be very time consuming. Side note: Mp3tag should change the filename of the .lrc files when you change the filename of the song (.flac, .opus etc.) files.
a) One way is to install and use lrcget to automatically batch download lyrics and then copy the generated .lrc files to the 5 Musixmatch folder.
b) Another way is to use the lrclib website which where lrcget fetches the lyrics. You can fine tune the search by yourself. Then you can copy the result to a text file with the same filename of the song you searched and use the .lrc extension. i.e. The Script; will.i.am - Hall of Fame [2012].flac should have a The Script; will.i.am - Hall of Fame [2012].lrc lyrics file.
c) One last way is to browse Genius website (or any other lyrics website) which has only unsynced lyrics and copy-paste the lyrics to an appropriate .lrc file you will create.
Once you have all the lyrics files together, you'll have to batch edit them.
Open all of the files in Notepad++ (just select them all and drag them inside the window). Turn on the document list from notepad to have a better view of the files in a list. Mp3tag and many music players that support lrc files seem to have some issues with extra linebreakes and the way linebreaks are created. On windows \r\n is used for a line break, while linux (and mp3tag/music players seem to prefer) \n. To fix this open the replace dialogue (ctrl+h) on Notepad++. Enable only the Wrap around, Transparency and Extended (search mode) options. In the Find what field we put the text we want to detect and in the Replace with we put the text which we want our detected text to be replace with.
We will run a series of replacements (follow the order):
a) Replace ] with ]
b) Replace ]\n with ]♪\n
c) Replace \n\r with ♪
d) Replace \r\n\r\n with \r\n♪\r\n
e) Replace \n\n with \n♪\n
f) Replace \r\n with \r\n♪
This should fix most of the lyrics. Save them all (you can try closing the Notepad++ window and press enter on each confirmation dialogue which will ask to save the file before closing.
After all these, in Mp3tag run the 7. Synced Lyrics from lrc action. It will copy the contents of the .lrc files in the songs (inside the SYNCEDLYRICS, LYRICS and UNSYNCEDLYRICS tags). The UNSYNCEDLYRICS tag will have the timestamps removed.
Note: Any songs that don't have an .lrc file will return an error window, just ignore it.
-
In Mp3tag run the 8. Export artist and title for genius and backup year and cover action. This exports a list containing only the artists and the titles of the songs in a .csv file which will later be used by our python script. It also copies the YEAR (date) tag in the DEEZER_YEAR tag. Lastly, it exports the cover images in the Images/Old folder, while adding the _Deezer suffix to each file created.
-
Time to use the script we made: Select a random song, right-click, go to Tools and select the Genius Data Fetcher tool. It will open a cmd window and run the python script. This may take a while. It will start writing the geniusdata.txt inside the Texts folder.
It may stop at a random point. If this happens, copy the contents of the geniusdata.txt to a separate text file, delete the geniusdata.txt and remove any artists-titles the script managed to scan on genius. You want to retain the order of the songs as they are in the titleartistlist.csv list, so be careful when you merge the data from interrupted uses of the script (create a geniusdata.txt with the same order of songs as they are on titleartistlist.csv).
-
Paste the contents of geniusdata.txt inside LibreOffice Calc (or any kind of Excel program), read them as csv and select Tab as the only delimiter. The script may not have been 100% accurate and it may has mistaken songs for different songs. To fix this, check if the genius artists and title match the last 2 columns which contain the artists and the title we used as input. If there is a mismatch set the date of this song to 0000-00-00. This will prevent from wrong info to appear later in the YEAR tag.
-
In Mp3tag select all the songs and right-click->Convert->Text file-Tag. In the filename field puth the path of the geniusdata.txt. In the format string field put this text: %genius_year% %genius_id% %genius_url% %genius_art_url% %temp%. This reads the values from the text file we created and pastes them in the appropriate tags of our songs.
-
In Mp3tag select all songs and run 9. Merge, trim, and genius Year action. This tidies up the YEAR and GENIUS_YEAR tags (merges any duplicate fields, converts the text None to 00 (redundant) and it adds a leading zero to the month and year fields of the genius year). Then it compares the GENIUS_YEAR and DEEZER_YEAR. If the Genius_year starts with 0 (meaning that the date is 0000-00-00) or the month and the days are zero (year-00-00) and the deezer_year has more info than the year (meaning if it isn't simply 4 digits without month-day) and the absolute difference of the years between deezer and genius years are less than one year, it will keep the old, DEEZER_YEAR in the YEAR field, otherwise the GENIUS_YEAR is written on the YEAR field.
-
In Mp3tag select all songs and run 4. Filename format artist - title [year]( -- [YT]) action. This updates the year in the filename since it probably changed in the step above (19).
-
Use any of the Genre adding [YT], Genre adding [Instrument@l], Genre adding [Gr33k], Genre adding [8b@ll], Genre adding [Nitr0], Genre adding [F@vourites] actions if you deem it necessary. Afterwards run the 6. Replace hiphoprap and Capitalise Genre action just to make sure things are properly capitalised.
I decided not to use playlists anymore, but instead to add in the GENRE tag, names of playlists or (custom) attributes. This way, if the player supports multiple genres, you ~don't need playlists anymore! You just pick the genre(s) you want. I put my custom Genres in [] to differentiate them from the rest. You can add more if you know how edit my code in the appropriate actions above. The code I've written checks if the Genre I add already exists in the GENRE tag (it reads all the genres in the tag by mentioning that the separator is \\) and if it doesn't, then it adds the new Genre, along with the rest that already existed.
[YT]: songs that I downloaded from Youtube
[Instrument@l]: songs that are purely instrumental
[Gr33k]: songs that are of Greek language (you can do something similar for other languages)
[8b@ll] and [Nitr0]: names of previous playlists
[F@vourites]: songs that I would otherwise add (or already have added) in my spotify/etc. favourites category
-
In Mp3tag select all songs that got donwloaded from Deemix (or Deezer) and run the Source to Deezer action. This writes the word Deezer in the SOURCE tag. You can edit this if you want to use it with different sources or simply just ignore this step.
-
To make sure the correct separators are used, use the ; to double inverted slash actions. I have decided to use the double inverted slash \\ as my separator in my .flac files when a tag field has multiple tags (for example the Artist tag in 2Pac; Roniece - Changes [2012].flac will be 2Pac\\Roniece. Mp3tag interpretes \\ as separator for multiple tags and if you check the extended tags of the song, you'll see the Artist tag twice). Many music players seem to get along with that, so I kept it.
-
Time to convert our songs to .opus (we will also keep the .flac files as well).
(!Attention!) There may be an issue with deleting/editing the .opus files on windows, this can be somewhat fixed by following one of the two soltions provided in the comments here on reddit (web archive link).
To do this we must open and set up fre:ac:
Go to Options->General settings setup:
a) Select Encoders tab:
-- Select the Opus Audio Encoder tab. Configure the encoder and leave everything default apart from Bitrate and file extension. Set the Bitrate to 128 and the extension to .opus. Click OK and go back.
b) Select Output files tab:
-- In the output folder put the path of the 6 Opus folder we created.
-- In the output filenames enable only the Use <filename> when title info is not available, Append sequential numbers to otherwise identical filenames and Allow Unicode characters. In the Filename pattern field put <filename> (this prevents fre:ac from creating new folder for each song, as the default behaviour would do).
And we're done! Fre:ac is set up.
Go to File->Add->Audio file(s) and add all the .flac files (this could also be done differently). Start the encoding process with the "play" button and wait.
-
Now we must put our songs in the correct folders:
-- Move all the .flac files and the .lrc files in the Flac folder we created inside the Final folder.
-- Move all the .opus files in the Opus folder we created inside the Final folder.
-- Copy-paste all the .lrc files in the Opus folder we created inside the Final folder.
-
Now we will transfer any tags to the .opus files that went missing during the conversion.
-- Open Mp3tag and choose the Flac folder path. Make sure songs are sorted alphabetically (from a to z), select all and then copy (ctrl+c).
-- In Mp3tag choose the Opus folder path. Make sure songs are sorted alphabetically (from a to z), select all and then paste (ctrl+v).
-
The last necassary (or not so necassary) step! In Mp3tag choose the Opus folder path. Select all songs and run the Merge and split tags action. This changes the separator from \\ to ; . The reason for that is because the music player on my phone doesn't recognize tag fields with multiple tags. Thus, I have to convert the seperator to ; and this way Mp3tag will merge all multiple tags of the same type into a single tag. Then my music player will separate the tags by recognizing the ; separator.
WE DID IT! Our songs are properly organized and tagged!
As you may have noticed, we didn't use one action. I'll explain what it does:
Remove favourite genre: Removes the [F@vourites] genre. It searches if this genre exist and if yes it removes it. The only limitation I managed to find in my code is that if you somehow only have the [F@vourites] genre multiple times (>1 time) and you have no other genre in the tag field, it won't remove anything. It seems like a very rare case, but this is a bug.
Happy tagging!
Extras:
Music players:
-
For my Windows PC I use Quodlibet. It reads multiple tags very well and scans my .flac collection rather fast.
How to set it up:
-- I have enabled the following plug-ins: Advanced Prefernces, Application Information, Change Language, ListenBrainz Submission, Rescan Songs, Synchronized Lyrics, Unique Query, View Lyrics and Waveform Seek Bar.
-- In layout prefernces I'm using the Small Column layout and in the column content I use a custom set up with 3 genre columns. I'm using the Panned browser.
-- In preferences:
a) Song list tab:
- I've enabled
Jump to playing song automatically, Always allow sorting, Sort songs when tags are modified and Title includes version. I've selected the Artist, Title, Year, Genre and Length to be visible.
b) Browsers:
- I have disabled
Enable one-click ratings
c) Library:
- In the scan directories I have put the path of the
Flac directory and have enabled Scan library on start.
d) Tags:
- I have everything disabled: I have disabled
Auto-save tag changes and Save ratings and play counts in tags.
Generally, I don't want Quodlibet to alter my tags.
-
For my Android (13) device I'm using Symphony.
It has support for custom tag separators, but it doesn't read multiple tags. That's the reason why I'm separting tags with multiple items using ; .
It takes a long while to initially load the files when you first add the folder, but once it does it is very fast. It has many features, probably the most you can find on an open source android local music player. Plus, it's beauuuutiful.
Music scrobblers/listen history recorders:
I use ListenBrainz on my devices to record my listen history. I have integated my Spotify account too (I just have to find a way to import my whole Spotify listening history with the proper tools). I have installed it on my android device and I added my music players to the list of apps to watch and record. On pc I have the built-in listenbrainz plugin enabled in Quodlibet. One issue or feature (depending on how you see it) with this platform is that it makes publicly available your listening history with detailed timestamps (date and time with minute resolution). That's why I am using a kinda random nickname. This helps you find people with similar tastes.
This is the end of the guide! I hope it helped you somehow or gave you any ideas for your workflow!
Yeah, (among others) I really want to learn git to sync dotfiles and the nixos configs
(Hopefully I'll probably have tranitioned to nixos in a few months. If I get good enough and somehow build a nas, I might use nixos instead of debian in the server too.)