Clustering
​
Finally, we used Clustering to identify any common characteristics between popular songs.
Both principal components analysis (PCA) and k-means were used in this clustering analysis.
Each song had multiple (11) dimensions, which were each of the audio features. PCA was used to reduce the dimensionality, which would help reduce the noise in the k-means clustering algorithm. PCA was used to essentially standardize and whiten the data. In this analysis, 9 dimensions were dropped.
K-means clustering was then used to find the natural groupings of songs. A Kaiser-Meyer-Olkin test was used to test the suitability of the data. With a KMO value of 0.596 the sample adequacy is almost mediocre. It indicates that our data is not the most suitable for clustering, meaning we can not heavily rely on the outcome of the k-means clustering.
​
The code used for the clustering and other components can be found here.
Since the PCA retained components with the highest variance, the clusters will be more visible as they will be the most spread out. PCA 1 accounts for over 25% of the variance while PCA 2 accounts for approximately 13% of the variance.
The elbow method suggested 3 groupings.
The second grouping (named as cluster one) is the largest one, including songs such as Rolling in the Deep by Adele, Ain't No Mountain High Enough by Diana Ross and Baby Got Back by Sir Mix-A-Lot. It contains, on average, the shortest songs with the highest danceability of 0.699 on a 0 to 1 scale. Cluster 1 has the lowest chance of being acoustic with a confidence of 0.107 and the highest average time signature of 4.005 over 4 (all songs bar 3 have 4/4 time signature). It also has the highest speechiness of 0.087.
The first group (named as cluster 0) includes songs such as I’ll Make Love to You by Boyz II Men, If I Ain’t Got You by Alicia Keys and Perfect by Ed Sheeran. Cluster 0 is the smallest, containing only 30 songs. They had the lowest danceability and energy, with an average of 0.494 and 0.369 respectively on a 0 to 1 scale. These songs also had the highest chance of being acoustic at an average confidence measure of 0.528 on a 0 to 1 scale. They also had the highest average tempo of 120.548 bpm. This cluster is also the only one with an average time signature of 3 over 4 - perhaps the defining factor of this cluster.
​
The third and last grouping (named as cluster two) contains 322 songs, including ABC by The Jackson 5, Rosanna by TOTO and Love Yourself by Justin Bieber. Cluster 2 had the, on average, longest songs with the slowest average tempo of 114.626 bpm. Cluster 2 songs have an average danceability of 0.584 on a 0 to 1 scale. All have a time signature of 4/4.
Adding Context to the Clusters
Time Signature
Only 33 songs from our dataset of 989 songs have a time signature that is not 4/4 - this is 3.3%. Looking through an electrophysiological lens, a binary metric organization such as 2/4 or 4/4 is arguably the human default system for Western listeners. There are faster brain responses for disruptions to binary structures, suggesting they are easier to process, and hence to enjoy (Potter, Fenwick, Abecasis and Brochard, 2009).
Release Year
​
These three box plots highlight the spread of release years for each cluster (0,1,2). 75% of the songs in Cluster 0 are between 1978 and 2001. This is a large range, potentially due to this cluster being the smallest one (30 songs).
In Cluster 1, 75% of the songs were released between 1988 and 2010. This can be classified as the 90s and 00s. The 90s was a period that could not necessarily be defined - it saw the continuation of dance-pop and RnB from the 80s and the rise of grunge with bands like Nirvana. More importantly, in 1991, the Billboard charts changed to reflect sales figures rather than record salespeople’s projections, making them more diversified. The 2000s saw the introduction of online music services, such as Napster, increasing the rate at which music could be shared and consumed. The same diversity of the 90s continued into the 2000s with artists like Ne-Yo and Katy Perry both competing for the top spots (Gopalan, 2019).
Cluster 2 focuses between the mid 70s and the mid 90s - 75% of the songs were released between 1976 and 1994. The 70s and 80s saw the popularity of rock rise, with bands like the The Police and Eagles. With MTV launching in the 80s, it defined culture and music visually (Ortiz, n.d.).
Acoustic-ness, Length and Tempo
This scatterplot highlights that Cluster 1 songs are very unlikely to be acoustic. These songs are predominantly from the 90s and 00s, which corresponds to the rising popularity of electronic (dance) music and also its influence in pop music (Feinstein and Ramsay, n.d.). Taking songs that aren’t acoustic as electronically-based, a reason why electronic music would be popular (with Cluster 1 constituting over 60% of our dataset) is due to the unique complexities and varied sonic textures it can introduce that aren’t available with traditional instruments. Cluster 2 songs are scattered throughout the acoustic scale on the plot, but they tend to be more likely to be acoustic. Acoustic songs provide the break and diversity perhaps needed in an electronic-filled music scene.
The length and tempo of songs stays consistent throughout clusters and acoustic confidence measures - with some anomalies.
Danceability, Energy and Tempo
This scatterplot shows a positive association between energy and danceability - the more energy a song has, the more suitable it is for dancing.
​
Majority of the top Billboard songs, and those in cluster 1, in the dataset have high energy and high danceability. Assuming that popular songs are those that make the listener feel happy, there is evidence that high energy songs (upbeat) boost one’s mood (Bergland, 2018). Music can be more effective at eliciting positive emotions than a visual stimulus, especially when attached to personal memories.
In regards to tempo, as seen by the size of each scatter point, lower tempos are considered low energy and danceable songs. Higher tempo means that there are more beats per minute, which - according to MRIs - activate motor areas of the brain (Kim, 2015). This causes us to move to the rhythm (read: dance), hence fast tempo songs are linked to higher danceability, which is a happy state, proving to be a potential justification for popularity.