Finally, the dataset is subjected to a cluster analysis based on the X-means algorithm, an extension of the well-known K-means algorithm that incorporates the use of information criteria to determine the appropriate amount of clusters (or value for K). Including all (150) features in the cluster analysis results in an unclear clustering consisting of a lot of clusters without distinct contents.
This leads to the decision of building different models, each focusing on a specific set of attributes or features. The first model is based on what The Echo Nest calls general features and results in six clusters. Upon investigation of the resulting cluster centres, it becomes clear that only some attributes lead to a certain distinction between clusters. Some clusters can be labelled as containing calm, quiet songs based on the attributes tempo and loudness but overall the content of the different clusters is not clear. This is further elaborated on in the visualisation of these clusters (as can be seen below). The graph shows no clear distinction between the different groups. Following this result, the next three models will be based on a set of variables that allows the user to form an idea of the resulting clusters upfront before applying the algorithm.
The first model to use this technique is based on twelve attributes relating to timbre. Applying the X-means algorithm based on these attributes leads to clear distinction containing only two clusters. The two cluster centres show significantly different values for all attributes and allow a clear visualisation. The model concerning timbre is accompanied by three graphs. The first graph plots T1mean against T2mean, the second T1mean against T6mean and the last T2mean against T10mean. All of these graphs clearly show the differences in both clusters and visualise how the relevant attributes are related to each other.
The following model tries to separate the dataset into popular and non-popular songs and uses the following six attributes: highest position, number of weeks, facebook likes, artist familiarity, artist hotness and song hotness. The clustering results in two clusters and the values for the centres clearly show that one cluster contains all songs that are considered popular and the other contains all songs that are considered non-popular. Visualising the clusters on a graph plotting the number of weeks against the highest position shows that positions 1 to 4 are almost exclusively reserved for songs from the popular cluster. Positions 6 to 10 on the other hand are mostly populated by songs from the non-popular cluster. Position 5 contains a mixture of both clusters and here the membership is mostly determined by other attributes like facebook likes, hotness and familiarity.
The final model uses all attributes that are related to the beats of a song. These include the tempo (beats per minute) of a song, the danceability and some statistical moments of the beatdif. Clustering based on these attributes results in four clusters. In general, these clusters can be divided into three categories. The first category is defined by a low value for tempo and a low value for danceability; the second consists of two clusters and is defined by a medium value for tempo and high values for danceability. The last category shows very high values for tempo but again low values for danceability. When visualising these clusters on the graph, it becomes clear that tempo and danceability are related to each other according to a pattern that resembles a banana. Extreme values for tempo (either extremely low or extremely high) result in a low value for danceability. Average values on the other hand, lead to a higher value for danceability. The second visualisation corresponding to this model highlights the negative relation between tempo and beatdif.