12
\$\begingroup\$

I was bored so I wrote a program in Mathematica to download all posts from the first 10 front pages (155 posts) and parse all entries to gather data from the languages and to make a top-list for code golfing.

The place is determined by the quotient of total bytes by posts, so it can be interpreted as bytes/post.

Here is the list.

Or, if you want to search for your language, have an alphabetical ordered one.

However, more interesting in my opinion is the comparison of the top 15 most popular languages by posts:

enter image description here

Also interesting are some graphs of the data:

enter image description here

enter image description here

enter image description here

The next graph is interesting. We see, that there is a clear gap between the data. I don't really know how this happens. (It reminds me very much of Sloane's Gap)

enter image description here

The code I used is far from perfect but does its job.

LoadCodeGolfPostLinks[index_]:=Module[{linkList,refinedLinkList},
linkList=Import["http://codegolf.stackexchange.com/questions?page="<>ToString[index]<>"&sort=votes","Hyperlinks"];
refinedLinkList=Select[linkList,If[StringLength[#]>44,StringTake[#,44]=="http://codegolf.stackexchange.com/questions/",False]&];
Select[refinedLinkList,(MatchQ[URLParse[#]["Path"],{"","questions",a_String?(IntegerQ@ToExpression[#]&),_String}])&]
]
LoadCodeGolfPostSource[link_]:=Module[{sideSource,sideMax},
sideSource=Import[link,"Source"];
sideMax=Max[{0}~Join~(ToExpression/@StringCases[sideSource,"title=\"go to page "~~a_?(IntegerQ[ToExpression[#]]&)~~"\""->a])];
If[sideMax>1,
sideSource=StringJoin[{sideSource}~Join~Table[Import[link<>"?page="<>ToString[i],"Source"],{i,2,sideMax}]]
];
sideSource
]
LoadEntries[source_]:=StringTrim/@(StringCases[source,"<div class=\"post-text\" itemprop=\"text\">"~~Whitespace~~("<h1>"|"<h2>"|"<h3>")~~e:Repeated[Except["<"|"\n"]]~~("</h1>"|"</h2>"|"</h3>")->e])
ParseEntrie[entrie_]:=Module[{languageNames,byteCount,centrie},
languageNames=(StringCases[entrie,Repeated[Except[" "|"\t"|","]]]);If[Length[languageNames]<1,
Null,centrie=StringDrop[entrie,StringLength[languageNames[[1]]]];
byteCount=ToExpression@(SelectFirst[StringCases[StringDelete[centrie,{"(",")"}]<>" bytes",count:Repeated[DigitCharacter]~~Whitespace...~~("bytes"|"characters"|"chars")->count,IgnoreCase->True],If[SyntaxQ[#],IntegerQ[ToExpression[#]],False]&]/.Missing["NotFound"]->(-1));If[IntegerQ[byteCount]&&byteCount>=0&&byteCount<10000,{StringTrim[StringTrim[languageNames[[1]],{":"}]],byteCount},
Null
]
]
]
MergeLC[lci_]:=Flatten/@(lci//.{"Wolfram"->"Mathematica","Brainf***"->"brainfuck","Brainf*ck"->"brainfuck","C++11"->"C++","C++14"->"C++","Python3.4.0b2"->"Python","AHK"->"AutoHokey","Windows"->"Batch","Bat"->"Batch","Visual"->"VBA",{a___,{"C/C++",b_},c___}:>{a,{"C",b},{"C++",b},c},{a___,{"C++/C",b_},c___}:>{a,{"C",b},{"C++",b},c}})
CountLC[lci_]:=Module[{languages,lMatches},
languages=DeleteDuplicates[lci[[All,1]]];
Function[{l},
lMatches=Select[lci,StringMatchQ[#[[1]],l,IgnoreCase->True]&];
{l,Total[lMatches[[All,2]]],Length[lMatches]}
]/@languages
]

Download code:

Monitor[
step="download links";
frontPages=10;
ll=Flatten[Table[step="download links n:"<>ToString[i]<>"/"<>ToString[frontPages];LoadCodeGolfPostLinks[i],{i,1,frontPages}]];
step="download source";
ss=Table[step="download source n:"<>ToString[i]<>"/"<>ToString[Length[ll]];LoadCodeGolfPostSource[ll[[i]]],{i,1,Length[ll]}];
,step]

Parsing code:

Monitor[
step="load entries";
ee=Flatten[LoadEntries/@ss];
step="parse entries";
lce=DeleteCases[ParseEntrie/@ee,Null];
step="merge language counts";
lc=MergeLC[lce];
step="count languages";
result=CountLC[lc];
step="calculate quotients";
quotientResult={#[[1]],#[[2]],#[[3]],N[#[[2]]/#[[3]]]}&/@result;
step="sort results";
resultData=SortBy[quotientResult,#[[4]]&];
,step]
resultData=DeleteDuplicatesBy[resultData,ToLowerCase[#[[1]]]&];

If you have any questions, you are free to ask them. Perhaps I can answer it.

\$\endgroup\$
10
  • \$\begingroup\$ For the graph with the gap, how do you determine "place of language"? \$\endgroup\$ Oct 6, 2016 at 15:44
  • \$\begingroup\$ The place is determined by average bytes per post. I edited it in. \$\endgroup\$ Oct 6, 2016 at 15:46
  • 10
    \$\begingroup\$ If you want to compare languages by size, please restrict it to answers in the [code-golf] tag. Other challenge types (koth, code-challenge) are more often answered in languages such as python, java, and c, which would inflate their scores over golf/eso-langs. They don't need to be inflated even more than they naturally are ;) \$\endgroup\$
    – Geobits
    Oct 6, 2016 at 16:02
  • 3
    \$\begingroup\$ I'm pretty sure your Sloane gap comes from the fact that the bottom curve is languages there were used exactly once and the next curve is languages that were used twice. (And the next, thinner line for languages used three times and then the really popular languages are all the noise at the top because they're used so often that it inflated their total byte count). \$\endgroup\$ Oct 6, 2016 at 21:43
  • \$\begingroup\$ Related: meta.codegolf.stackexchange.com/q/5079/25180 \$\endgroup\$
    – jimmy23013
    Oct 11, 2016 at 3:29
  • \$\begingroup\$ Related challenge. \$\endgroup\$ Oct 11, 2016 at 11:30
  • \$\begingroup\$ Another related challenge and very nice graphs in this answer \$\endgroup\$
    – Sanchises
    Oct 14, 2016 at 10:43
  • 4
    \$\begingroup\$ This information isn't entirely useful if you're using the entire post to count bytes. There are plenty of answers which contain both a golfed and an ungolfed version, plus an explanation paragraph. \$\endgroup\$
    – mbomb007
    Oct 14, 2016 at 18:58
  • 2
    \$\begingroup\$ I'm voting to close this question as off-topic because it isn't actually a question. \$\endgroup\$
    – user45941
    Oct 17, 2016 at 22:32
  • \$\begingroup\$ am I reading this right? the posts aren't even parsed to exclude the non-code blocks? \$\endgroup\$
    – user60119
    Oct 18, 2016 at 23:41

0

Browse other questions tagged .