GNU Savannah - project statistics

Active Life-Time of projects hosted on GNU Savannah

• BZR information is missing, projects using BZR will show up as not-recently-active
• Some projects (e.g. GCC) host their code outside GNU Savannah. Such projects will show up as not-recently-active (since their repository on Savannah is empty or out-dated).
• Decomissioned GNU projects as still shown in this chart.
• In the data file (gnu-sv-projects.txt):
  X Axis is column 4 (code_min_year) = year of first commit.
  Y Axis is column 5 (code_max_year) = year of most recent commit (as of Sep-2014).
  Year is a floating-point value, representing the year (the integer part) and day/month (fractional part). Pseudo-code is:

  t = unix-time of last commit (seconds since epoch)
  struct tm* tmp = gmtime(t)
  year = tmp->year+1900 + tmp->yday/366;
  

  2012.01 is approximately 01-January-2012
  2013.50 is approximately 30-June-2013;
  2014.75 is approximately 30-September-2014
  

• GNU Projects without code commits:

  awk '$2=="gnu" && $3=="NA" { print $1 }' gnu-sv-projects.txt

  
  (NOTE: some projects host their code elsewhere).
• GNU Projects dormant for 5 years or more (print project,vcs, first commit, last commit):

  awk -v OFS="\t" '$2=="gnu" && $5!="NA" && $5<2009 { print $1, $3, $4, $5 }' gnu-sv-projects.txt

• Projects which started more than 20 years ago:

  awk -v OFS="\t" 'NR==1 || ($4!="NA" && $4<1994) {print $1,$4,$5,$6}' gnu-sv-projects.txt
  

• Non-GNU projects which where updated in the last two years, sorted from oldest to newest ('gpsd' going on its 17th year...)

  awk -v OFS="\t" 'NR==1 || ($2=="non-gnu" && $5!="NA" && $5>=2013) {print $1,$3,$4,$5,$5-$4,$6}' gnu-sv-projects.txt | ( sed -u 1q ; sort -k5nr,5 ) | less
  

• The in make-gnu-sv-project-plots.r R script, the plot is plot(data$code_min_year,data$code_max_year, ... .

Homepages and Downloads

• The "Homepage" and "Downloads" URLs are based on each project's GNU Savannah's page.
• Most GNU projects have a functional homepage, but few do not. Examples:
  libiconf (gives HTTP 404 on gnu.org)
  macchanger (gives HTTP 404 on an external server)
  Even if the project has another well-known homepage, this is the URL given on GNU Savannah when user click "homepage" - the user will still get an HTTP error.
• "Downloads" (also called 'releases') are links to a FTP listing of downloadable tarballs.
  To be considered 'valid', the page must exist, and must contain at least one tarball with a name similar to the project name. Detection is not perfect, so incorrectly marked projects are possible.
• Some GNU projects points to invalid 'downloads' directoies (gnuspeech); Others have a special README file on ftp.gnu.org (w3), but GNU Savannah still gives an invalid download URL for them (w3 Savannah Link).
  
• Many Non-GNU projects never made a public release (e.g. acml).

• In the data file, the following fields are relevant:
  Column 14: Homepage URL (as shown on GNU Savannah)
  Column 15: Downloads URL (as shown on GNU Savannah)
  Column 16: 1 or 0; 1 = Homepage is valid (not HTTP 404 or invalid host).
  Column 17: 1 or 0; 1 = Download is valid (not HTTP 404, and contains tarballs with same name as project).
  
• Count valid/invalid hompages & downloads per project:

  tail -n +2 gnu-sv-projects.txt | cut -f2,16,17 | sort | uniq -c
  

• Same as above, with GNU Datamash:
  

  datamash -sH -g 2,16,17 count 1 < gnu-sv-projects.txt
  

• GNU Projects with invalid homepage:

  awk '$2=="gnu" && $16==0 { print $1, $14 }' gnu-sv-projects.txt
  

• GNU Projects with invalid Downloads page::

  awk '$2=="gnu" && $17==0 { print $1, $15 }' gnu-sv-projects.txt
  

• GNU Projects which don't use www.gnu.org as main homepage:

  awk -v OFS="\t" '$2=="gnu" && $14 !~ /www\.gnu\.org/ { print $1, $14}' gnu-sv-projects.txt
  

• See also the hp_dl variable in make-gnu-sv-project-plots.r R script.

Active Projects Types and VCS

• VCS and project types (for all projects):

  awk 'NR>1 {print $2, $3}' gnu-sv-projects.txt  | sort | uniq -c
  

• Same with GNU Datamash:

  datamash -sH -g 2,3 count 3 < gnu-sv-projects.txt
  

• VCS and project types for recently updated projects (similar to above chart):

  awk -v OFS="\t" 'NR==1 || ($5!="NA" && $5>=2013)' gnu-sv-projects.txt | datamash -sH -g 2,3 count 3
  

• List projects for each type/vcs:

   awk -v OFS="\t" 'NR==1 || ($5!="NA" && $5>=2013)' gnu-sv-projects.txt | datamash -sH -g 2,3 count 3 collapse 1 | less -S