Adding organization repos to Travis-CI: Part I

It was a nice FridayPorras and I were at the BeBanjOffice and  we were working on adding organization support to Travis-CI.

First, we needed to find where the user repos were being fetch. Said and done, we found that travis is using the ServiceHookController to ask the server for repos (and to update them too, but don’t hurry, we’ll get there in part II). The controller was using the github_repositories method in the User model. Finally, we arrived to Travis::GithubApi, where all our questions found answers.

repositories_for_user just relied on Octokit#repositories for getting the repos, so we added all the code after the plus sign. You might think that this code lacks the proper security, is wrong and that it shouldn’t be there (and damn it, you’re right!) but there is no easy way to retrieve only the repos where the user has admin privileges (we wasted hours on this and a quick message to Github confirmed it).  Basically, it asks for all organizations for the given user (beware! You need to make your membership public, otherwise it won’t work) and then flat_map them to the repos in the given organization (whether the user can administrate them or not).

Later, the each block was added so we could know whether the repo belongs to the user or to an organization.

Stop! Hammertime!

We needed a way to let the user know that a repo actually belongs to an organization, so we added that info to the template (that {{#if content.organization_name}} is ours :D) and added some style so it could look this awesome:

Travis CI - Distributed build platform for the open source community

We had some tough time trying to get it vertically centered, but after realizing that Github has its tags the same way as we do, we stop trying to fix it. And they still look awesome.

More coming in part II.

Writing your own JRuby extension. Part II: Creating your first class.

More Mormon Matryoshki
What’s the point of coding a JRuby extension if you don’t create classes? Well, I cannot think of any case, but if you find one, please, let me know.

Do you remember Java’s classes hierarchy? If so, you’ll realize  that Java objects inherit from java.lang.Object by default but that is not actually what we need. But how can we tell JRuby that our object inherits from Object, the king of Ruby world? The answer is so simple: just extend RubyObject! (There is a RubyBasicObject too) Taken from Nokogiri::XML::Node:

Of course, you can extend any other class, as long as it is a “Ruby object”. For example, Nokogiri::XML::Document extends from Nokogiri::XML::Node, and we do not need to do anything special to reflect it, just extend XmlNode like XmlDocument does:

After talking a bit about hierarchy, let’s talk about Java constructors. At least two parameters are needed: a org.jruby.Ruby object and a org.jruby.RubyClass object. Being the importance of the former quite obvious, the reason for the latter may not be so clear. Let me show you some real world code: Nokogiri::XML::Node’s dup method. Take a look at the following test:

Both new and dup methods in subclass rely on Nokogiri::XML::Node’s. In here, you can see the this snippet of code:

The rb_obj_class method returns the class of an object, in this case, self. This way, the new node will be an instance of the same class as the original node. That’s the reason a RubyClass is needed in the constructor, in order to know which is the actual class being instantiated.

By the way, do not forget to call super with the Ruby and RubyClass objects.

Next step: creating methods.

Creative Commons License photo credit: quinn.anya

Writing your own JRuby extension. First problem.

Denial
Maybe, when requiring your just created extension, you get a LoadError. If it is the first time you require it, it is quite likely that you have not followed JRuby requiring conventions. If you want to know how require works, you can find the best documentation ever about it in the comment before org.jruby.runtime.load.LoadService class.

Creative Commons License photo credit: cesarastudillo

Writing your own JRuby extension. Part I: BasicLibraryService.

Note: not code in this post, but you can see the code in Github. Follow the links!

Writing a JRuby extension is very easy, but there are almost not post out there about it. As far as I know, there is only one, Ola’s. It is a really good tutorial indeed, but it lacks some details that might be not-that-easy to solve. Please, take some time to read it and, if some details are different, do follow Ola’s way.

Everything’s ready now, so let’s start talking about BasicLibraryService. If you take a look at Nokogiri4J sourcecode, in ext/java/nokogiri folder, you will see a NokogiriService.java file. NokogiriService implements BasicLibraryService. This interface consists only of the method basicLoad which receives a Ruby object.

We will use this method to define classes and methods in the Ruby world. For defining a module, defineModule method is used with the name of the module. After that, modules and classes under that module can be defined easily by using the methods defineModuleUnder, which takes the name as parameter, and defineClassUnder, which takes the name and few parameters more. Let’s dive into it.

defineClassUnder needs three arguments. The first one is the class’ name. The second, is the parent class. If you have defined it previously, just passed it,  otherwise use RubyObject by calling the method getObject on the Ruby instance. The third parameter is an ObjectAllocator. ObjectAllocators returns intances of the classes in Java world. When instantiating Nokogiri::XML::Comment in Ruby world, JRuby will ask the ObjectAllocator for an instance of the Java class. It passes a Ruby object and the RubyClass being instantiated to the allocate method in the ObjectAllocator (more on RubyClass in following posts).

Finally, we will need to define some methods. Easiest way is by using the defineAnnotatedMethods. It takes a Java class as parameter. For knowing what this method does, you need to know a bit more about @JRubyMethod annotation (more on it in following post, have you realized the “Part I” in the title?). As you define methods, you may need to undefine some in a subclass. So easy! Use the undefineMethod method, which takes the name of the method as parameter (surprisingly, it undefines a method by redefining it!).

Next time, Implementing your first class.

Helping Nokogiri. Take II

PIZZA PRO 3000
Nokogiri and Pizza, what else can you ask for?

Ok. My fault. Now, let’s go get some work done.

First, if you haven’t done it, read the previous post about helping nokogiri and forgot about the script and memory leak. It seems that there are more important issues, so let’s fix them first. First run jruby test/test_jruby.rb from Nokogiri root. You’ll see a lot of errors (27 by now) and failures (14). Choose one, and get it green. After that, send me a pull request.

Ok. That sounds simple, but what if  the number of errors or failures raises? The rule I use is simple: keep the sum of both numbers going down and having a failure is better than having an error.

On the other hand, if you take a look at test/test_jruby.rb, you’ll see that not every test is in there. There is a reason for that. Even keeping the number of test low, you get a lot of errors/failures. If that annoys me with just 50 failures, imagine if I had a couple of hundreds errors. When all is right, I’ll add some more to keep the fun on.

Photo by Paul Johnston.

Do you wanna help us with pure-Java Nokogiri?

First things first, if you wanna help, you’ll need to clone the git repo. Just:

git clone git://github.com/tenderlove/nokogiri.git
cd nokogiri
git checkout --track -b java origin/java

Install the dependencies. Just:

rake install:deps

Because it uses some native libraries, you’ll need to do that with MRI. Finally, you’ll need to generate some files, just run jruby -S rake java:spec. For having a hprof file, you’ll need to run this script with the following command:

jr -J-Xmx32m -J-XX:+HeapDumpOnOutOfMemoryError nokogiri_doc_frag.rb

-J-Xmx32m limits the heap space to 32 Mb, and the other options makes the JVM to write a hprof file when a OutOfMemoryError is thrown. After that, you can inspect that file with the profiler you can find in NetBeans.

In next post, I’ll comment where I think the problem is.

Final Status Update (or How to get Nokogiri in JRuby without FFI)

Hi, all,

sorry for the silence all these months, but I’ve been working hard on Nokogiri. First things first, it is not complete yet. Anyway, I’m gonna tell you how to build a gem and start working with it.

Wooden vice - sharpening saw
Creative Commons License photo credit: Matthew Byrne

Clone the repo and checkout the java branch.

git clone git://github.com/tenderlove/nokogiri.git
git checkout --track -b java origin/java

Next step: build the gem. As easy as jruby -S rake java:gem. Maybe, you need to install rexical and racc. jruby -S rake install:deps would do.

Finally, you have a gem in the pkg folder. Install it, and you’re done.

Updated: Take a look here if you wanna help.

Status Update: RMagick4J, Nokogiri, ruby2java and a possible MagickWand4J

It’s been long time since last status update, but there are some things to tell, so here I am.

Thankfully, this year I’m a GSoC student again (and my mentor is Tom too). The main part of my project would be porting Nokogiri to JRuby, so I haven’t code for RMagick for a while now.

Let’s start with the status update then.

Nokogiri

I’ve been working on Nokogiri for a while. I forked Charles’ repo in Github, and I’ve implemented some cool features. For example, today I got my XML::Reader implementation to pass all tests in test_reader.rb. I hope I’ll be able to make a release this month (cross your fingers).

On the other hand, I got my first patch accepted in Nokogiri’s main repo.

RMagick4J

Not to much work done here, sorry. I haven’t code anything for a while now. Migrating from mercurial to git is already planned, but before that I would like to do a few commits more. Anyway, I’m quite happy with this project. Some people are using it and reporting bugs (in the end, those little things are all that matters). What else can I ask for?

Please, if you find a bug, report it here.

MagickWand

Tim Hunter (creator of RMagick) released MagickWand recently. I’ve been considering porting it to JRuby too. I have to take a deeper look at the C code, but, by now, I think it could be a good way to lead RMagick4J development. If finally I port it, I will split RMagick4J in two projects (Magick4J and RMagick4J). This way, MagickWand4J and RMagick4J would share the same java codebase, as MagickWand and RMagick share ImageMagick.

ruby2java

Take a look here. Awesome, isn’t it? And as soon as I have some time to work on it, siesta will be out too…

P.D. By the way, no more personal stuff in this blog. That stuff is now here, and only in Spanish (sorry about that).

RMagick4J 0.3.7 Happy Birthmonth released.

RMagick4J 0.3.7 Happy Birthmonth has been released!

RMagick is a Ruby binding to ImageMagick and GraphicsMagick. RMagick4J implements ImageMagick functionality and the C portions of RMagick for use with JRuby.

Current stable version: 0.3.7
Project URL: http://kenai.com/projects/rmagick4j
Installation: gem install rmagick4j

New effects:

  • Charcoal
  • Edge
  • Implode
  • Negate
  • Normalize
  • Shade
  • Solarize
  • Wave

(I think that’s all that you need to use simple_captcha)

Major bugs resolved:

  • blur_image produced different images from RMagick.
  • Draw#push and Draw#pop didn’t work correctly in some cases.

Furthermore, RMagick4J does not depend on jhlabs library anymore (the gem has lost some weight!).

Thanks to:

  • Tim Hunter and the ImageMagick team (obviously).
  • JRuby community for its support.
  • Everyone that has reported a bug, ask for help in the #jruby channel or suggest a new library compatibility.

Please try out your applications with rmagick4j and help us provide feedback (even birthmonth cake!). It is our goal to make a fully-compatible implementation of RMagick4j in JRuby.

You have some image examples here: http://www.serabe.com/rmagick4j

This is a very special release, cause March is my birthmonth.

Happy Birthmonth to:

  • SuperTaz
  • everyone else whose birthmonth is march
  • and me!