The second survey data set (2a) has been collected and the form is now closed. I have gathered a first analysis of the results here.
For the second survey I received 105 responses, 74 of which were from Finnish respondents. In all there were participants with 18 different nationalities that were geographically spread wide, from the United States to South American countries, and Europe to Asia.
67% of the respondents were male and 23% female.
The largest age group of the respondents were 30 to 40 years old (39%), with the entire age range varying from 21 to 57 years.
The participants were asked to describe their skills in using digital devices on a five step range: very good, good, average, poor, or very poor. 68% of them described their skills as very good, 23% good, 8% average and 1% poor. They were also asked how often they found it difficult to understand the meaning of interface icons or buttons. This was also measured on a five step range: very rarely, rarely, occasionally, often, or very often. 28% of them replied very rarely, 40% rarely, 29% occasionally, 2% often and 1% very often.
The respondents were recruited from university students and staff, LinkedIn and Facebook GUI interest groups and my personal networks.
In general the demography was similar to the previous survey. The portion of male respondents was however 9% higher and the 30-40 age group was less dominant, 39% versus the previous 49%.
Classification task and lack of context
The respondents were asked to classify 20 interface icons as metaphoric, descriptive or not applicable. In addition the respondents were asked to shortly name what they thought each of the icons stands for, what is its meaning or function. The purpose of this question was to verify, that the user had correctly recognized the icon. Just like in the previous survey, some of the icons could be understood in a variety of ways, and others were not always recognized at all. One key factor here is the lack of context. For instance the magnifying glass was recognized as both a search and magnifying tool. Similarly, the cross icon that stands for closing or deleting, was also recognized as the symbol for irritating substances.
In order to compare the responses in a reliable way, these variations need to be eliminated as much as possible. Therefore, I have ignored all the responses where the user has either not recognized the icon, or has thought of an alternative meaning for it. Consider the cross icon. If a user understands it as a symbol for irritating substance, his or her response on the question: "Is the icon metaphorical or descriptive?", is not comparable to the other users, who recognized it as the close icon.
The total number of icon classifications was 2100 (105 participants x 20 icons). Out of these, there were 219 cases, where the icon had not been correctly recognized. Therefore, 89.6 percent of the data was valid, and this part of it can be analyzed further.
One of the hypotheses here was, that the icons that were classified as arbitrary in the first survey, should be classified as not applicable in the second survey, since there apparently is no logical relation between the icon and its object. The icons that were classified dominantly arbitrary were: stop, record, power, pause, eject, and close. All of these were indeed classified as dominantly not applicable. The only flaw in the logic was, that two additional icons (fast forward and rewind) were also classified as not applicable.
The icons that were classified as mostly descriptive were: print, calculator, folder, trash, copy, file, and play. Perhaps the most suprising result here was, that calculator, folder, and trash icons were seen as despcriptive rather than metaphoric. The calculator application is clearly a part of the classic desktop metaphor, but perhaps the icon can be also considered as descriptive since it looks similar to the application itself. Interestingly also the play icon was the only playback icon in this group, as all the others (stop, pause, rewind, fast forward, eject) were classified as not applicable.
The metaphoric classification group didn't offer as many deviations. The group consisted of the icons: search, save, paste, cut, and reload. What is suprising, is the fact that cut and paste were a part of this group, but copy was seen as a descriptive rather than metaphoric icon.
Although there are logical patterns in the classification data, it is also apparent that the respondents found the task of classifying icons according to this terminology very challenging. This was clear from the open feedback that was received from the respondents. It might also explain why the differentiation between the three classes is not so desicive. Certainly this type of classification can only be useful for a designer or researcher, who is more used to dealing with such logical relations, and even that can be questioned. The single most suprising hypotheses that can be drawn from the results was, that an application based on the desktop metaphor, like the calculator, can have an icon that is considered as descriptive, since the icon looks very similar to the application.
The aim of the this survey was also to evaluate whether a two level classification system described in figure 1 might be used as tool in the task of designing or selecting what icons to use in an interface. This system was described in more detail in the online article: An Experimental Approach on Icon Taxonomy and the poster article for HCI Toronto 2016: A Practical Approach to Icon Taxonomy that followed it.