Tomorrow I will attend the first Hackergarten at Paris.
This is an event where active open-source committers can met potential contributors and help them to submit a (first) contribution.
I’ll be here to help those who would like to contribute on JenkinsCI and Apache Maven.
To prepare this event I wanted to prepare a checkout of all sources and a local repository with all required dependencies to avoid to see contributors losing their time by downloading the earth with Maven or checking out Subversion (and to not be blocked if we had some network issues).
For Maven which is hosted on Subversion it is long but easy because we have a special SVN directory using svn:externals to checkout all trunks in one step : https://svn.apache.org/repos/asf/maven/trunks/
But for Jenkins with several thousands of plugins and thus git repositories it is less easy to clone all of them.
I saw that there was everything needed in github APIs but it was late and I didn’t want to start to write a new script (One teacher told me a long long time ago that it was a quality to be idle – do it with less and less you’ll have to maintain).
I asked on jenkinsci IRC channel and thanks to Jørgen P. Tjernø I had my reply quickly.
With Ruby 1.8 (default version on Macos) :
sudo gem install json
curl -s https://api.github.com/orgs/jenkinsci/repos | ruby -rubygems -e 'require “json”; JSON.load(STDIN.read).each {|repo| %x[git clone #{repo["ssh_url"]} ]}'
With Ruby 1.9+, the json library is by default thus you just use :
curl -s https://api.github.com/orgs/jenkinsci/repos | ruby -rjson -e 'JSON.load(STDIN.read).each {|repo| %x[git clone #{repo["ssh_url"]} ]}'
With that I’m able to clone all repositories from jenkinsci github organization.
Tomorrow I’ll just have to launch a command like this to download all dependencies :
find . -name pom.xml -maxdepth 2 -exec mvn -f {} -Dmaven.repo.local=./maven-local-repo dependency:go-offline \;
Update : As there are more than 30 repositories in JenkinsCI Github organization, I had to call the clone command several times by adding ?page=X (X=2,..) after the url https://github.com/api/v2/json/repos/show/jenkinsci to grab all results pages.
Update 2 : We can also use per_page to increase the number of entries returned (max=100). Example (ruby 1.8) :
curl -s "https://github.com/api/v2/json/repos/show/jenkinsci?per_page=100&page=2" | ruby -rubygems -e 'require "json"; JSON.load(STDIN.read)["repositories"].each {|repo| %x[git clone #{repo["url"]} ]}'
Suivez moi !!!
Tags
Apache Archiva Apache Maven Blog Buzz Communauté Conférence CRaSH DevOps devoxx FeedBurner Git GitHub Google Gradle Groovy Hudson Java Java User Group Jenkinsci Jenkins User Conference JugSummerCamp LesCastCodeurs MacOS X Leopard modélisme octo Octo Technology Puppet Sonatype Nexus Subversion Université du SI Unix USI Vagrant Veewee WordPressCatégories
- Actualité (33)
- Le blog (3)
- Ma vie (4)
- Technologie (22)
-
Recent Posts
- 1ère édition DevopsDays PARIS – PROMO Inside
- Gagnez votre place pour Devoxx France 2013
- Launch a Maven build with a temporary empty local repository
- Maven : Filter only a given type of files
- Il ne faut pas CRaSHer dans la soupe … au JugSummerCamp
- Vers l’infini et au delà …
- Maven vs Gradle au LavaJUG
- Devoxx France 2012 – Gagnez votre place
- En avril, la Jenkins User Conference pose ses valises à Paris
- USI 2012 : Payez votre place à moitié prix.
Recent Comments
- Arnaud Héritier on Clone all repositories from a github organization
- Antoine_h on Clone all repositories from a github organization
- Antoine_h on Clone all repositories from a github organization
- Ansgar on Launch a Maven build with a temporary empty local repository
- Adrien on Launch a Maven build with a temporary empty local repository
Dernière publication


Fun for a REST Json API to have pagination enabled by default as the webUI does :-/
thought i would just add, v2 api wasnt working for me
the updated command for the v3 api would be (for ruby 1.8.7):
curl -s https://api.github.com/orgs/jenkinsci/repos | ruby -rubygems -e ‘require “json”; JSON.load(STDIN.read).each {|repo| %x[git clone #{repo["ssh_url"]} ]}’
Note that i also use the ssh_url to clone the repo’s as the url now defaults to the api url which will not work and send these errors:
fatal: https://api.github.com/repos///info/refs not found: did you run git update-server-info on the server?
Thx a lot for this update.
That’s right that Github removed its v2 apis.
I will update my post.
Thx again
Nice,
I was looking for this kind of tool, for a while.
This is usefull to get all the GateIn source code, or all the ExoPlatform source code, to dig and search into it.
Thanks a lot.
For any reason, the shell command did not work on my machine, because of the inverted commas or so… so I modified it (only thequotation mark).
Here is the command, for Ruby 1.9+ :
curl -s https://api.github.com/orgs/jenkinsci/repos | ruby -rjson -e “JSON.load(STDIN.read).each {|repo| %x[git clone #{repo[\"ssh_url\"]} ]}”
In fact I found that I have a problem with my blog. It replaces in code simple quotes by ‘…’
I will try to find a solution ASAP