Florian Quèze

To content | To menu | To search

Mozilla

Posts related to Mozilla activities

Entries feed - Comments feed

Sunday, July 27 2014

Firefox is awesome

"Hey, you know what? Firefox is awesome!" someone exclaimed at the other end of the coworking space a few weeks ago. This piqued my curiosity, and I moved closer to see what she was so excited about.

When I saw the feature she was delighted of finding, it reminded me a similar situation several years ago. In high school, I was trying to convince a friend to switch from IE to Mozilla. The arguments about respecting web standards didn't convince him. He tried Mozilla anyway to please me, and found one feature that excited him.
He had been trying to save some images from webpages, and for some reason it was difficult to do (possibly because of context menu hijacking, which was common at the time, or maybe because the images were displayed as a background, …). He had even written some Visual Basic code to parse the saved HTML source code and find the image urls, and then downloaded them, but the results weren't entirely satisfying.
Now with Mozilla, he could just right click, select "View Page Info", click on the "Media" tab, and find a list of all the images of the page. I remember how excited he looked for one second, until he clicked a background image in the list and the preview stayed blank; he then clicked the "Save as" button anyway and… nothing happened. Turns out that "Save as" button was just producing an error in the Error Console. He then looked at me, very disappointed, and said that my Mozilla isn't ready yet.
After that disappointment, I didn't insist much on him using Mozilla instead of IE (I think he did switch anyway a few weeks or months later).

A few months later, as I had time during summer vacations, I tried to create an add-on for the last thing I could do with IE but not Firefox: list the hostnames that the browser connects to when loading a page (the add-on, View Dependencies, is on AMO). I used this to maintain a hosts file that was blocking ads on the network's gateway.
Working on this add-on project caused me to look at the existing Page Info code to find ideas about how to look through the resources loaded by the page. While doing this, I stumbled on the problem that was causing background image previews to not be displayed. Exactly 10 years ago, I created a patch, created a bugzilla account (I had been lurking on bugzilla for a while already, but without creating an account as I didn't feel I should have one until I had something to contribute), and attached the patch to the existing bug about this background preview issue.
Two days later, the patch was reviewed (thanks db48x!), I addressed the review comment, attached a new patch, and it was checked-in.
I remember how excited I was to verify the next day that the bug was gone in the next nightly, and how I checked that the code in the new nightly was actually using my patch.

A couple months later, I fixed the "Save as" button too in time for Firefox 1.0.

Back to 2014. The reason why someone in my coworking space was finding Firefox so awesome is that "You can click "View Page Info", and then view all the images of the page and save them." Wow. I hadn't heard anybody talking to me about Page Info in years. I did use it a lot several years ago, but don't use it that much these days. I do agree with her that Firefox is awesome, not really because it can save images (although that's a great feature other browsers don't have), but because anybody can make it better for his/her own use, and by doing so making it awesome for millions of other people now but also in the future. Like I did, ten years ago.

Friday, July 25 2014

Converting old Mac minis into CentOS Instantbird build slaves

A while ago, I received a few retired Mozilla minis. Today 2 of them started their new life as CentOS6 build slaves for Instantbird, which means we now have Linux nightlies again! Our previous Linux build slave, running CentOS5, was no longer able to build nightlies based on the current mozilla-central code, and this is the reason why we haven't had Linux nightlies since March. We know it's been a long wait, but to help our dear Linux testers forgive us, we started offering 64bit nightly builds!

For the curious, and for future reference, here are the steps I followed to install these two new build slaves:

Partition table

The Mac minis came with a GPT partition table and an hfs+ partition that we don't want. While the CentOS installer was able to detect them, the grub it installed there didn't work. The solution was to convert the GPT partition table to the MBR older format. To do this, boot into a modern linux distribution (I used an Ubuntu 13.10 live dvd that I had around), install gdisk (sudo apt-get update && sudo apt-get gdisk) and use it to edit the disk's partition table:

sudo gdisk /dev/sda
Press 'r' to start recovery/transformation, 'g' to convert from GPT to MBR, 'p' to see the resulting partition table, and finally 'w' to write the changes to disk (instructions initially from here).
Exit gdisk.
Now you can check the current partition table using gparted. At this point I deleted the hfs+ partition.

Installing CentOS

The version of CentOS needed to use the current Mozilla build tools is CentOS 6.2. We tried before using another (slightly newer) version, and we never got it to work.

Reboot on a CentOS 6.2 livecd (press the 'c' key at startup to force the mac mini to look for a bootable CD).
Follow the instructions to install CentOS on the hard disk.
I customized the partition table a bit (50000MB for /, 2048MB of swap space, and the rest of the disk for /home).

The only non-obvious part of the CentOS install is that the boot loaded needs to be installed on the MBR rather than on the partition where the system is installed. When the installer asks where grub should be installed, set it to /dev/sda (the default is /dev/sha2, and that won't boot). Of course I got this wrong in my first attempts.

Installing Mozilla build dependencies

First, install an editor that is usable to you. I typically use emacs, so: sudo yum install emacs

The Mozilla Linux build slaves use a specifically tweaked version of gcc so that the produced binaries have low runtime dependencies, but the compiler still has the build time feature set of gcc 4.7. If you want to use something as old as CentOS6.2 to build, you need this specific compiler.

The good thing is, there's a yum repository publicly available where all the customized mozilla packages are available. To install it, create a file named /etc/yum.repos.d/mozilla.repo and make it contain this:

[mozilla]
name=Mozilla
baseurl=http://puppetagain.pub.build.mozilla.org/data/repos/yum/releng/public/CentOS/6/x86_64/
enabled=1
gpgcheck=0

Adapt the baseurl to finish with i386 or x86_64 depending on whether you are making a 32 bit or 64 bit slave.

After saving this file, you can check that it had the intended effect by running this comment to list the packages from the mozilla repository: repoquery -q --repoid=mozilla -a

You want to install the version of gcc473 and the version of mozilla-python27 that appear in that list.

You also need several other build dependencies. MDN has a page listing them:

yum groupinstall 'Development Tools' 'Development Libraries' 'GNOME Software Development'
yum install mercurial autoconf213 glibc-static libstdc++-static yasm wireless-tools-devel mesa-libGL-devel alsa-lib-devel libXt-devel gstreamer-devel gstreamer-plugins-base-devel pulseaudio-libs-devel

Unfortunately, two dependencies were missing on that list (I've now fixed the page):
yum install gtk2-devel dbus-glib-devel

At this point, the machine should be ready to build Firefox.

Instantbird, because of libpurple, depends on a few more packages:
yum install avahi-glib-devel krb5-devel

And it will be useful to have ccache:
yum install ccache

Installing the buildbot slave

First, install the buildslave command, which unfortunately doesn't come as a yum package, so you need to install easy_install first:

yum install python-setuptools python-devel mpfr
easy_install buildbot-slave

python-devel and mpfr here are build time dependencies of the buildbot-slave package, and not having them installed will cause compiling errors while attempting to install buildbot-slave.

We are now ready to actually install the buildbot slave. First let's create a new user for buildbot:

adduser buildbot
su buildbot
cd /home/buildbot

Then the command to create the local slave is:

buildslave create-slave --umask=022 /home/buildbot/buildslave buildbot.instantbird.org:9989 linux-sN password

The buildbot slave will be significantly more useful if it starts automatically when the OS starts, so let's edit the crontab (crontab -e) to add this entry:
@reboot PATH=/usr/local/bin:/usr/bin:/bin /usr/bin/buildslave start /home/buildbot/buildslave

The reason why the PATH environment variable has to be set here is that the default path doesn't contain /usr/local/bin, but that's where the mozilla-python27 packages installs python2.7 (which is required by mach during builds).

One step in the Instantbird builds configured on our buildbot use hg clean --all and this requires the purge mercurial extension to be enabled, so let's edit ~buidlbot/.hgrc to look like this:
$ cat ~/.hgrc
[extensions]
purge =

Finally, ssh needs to be configured so that successful builds can be uploaded automatically. Copy and adapt ~buildbot/.ssh from an existing working build slave. The files that are needed are id_dsa (the ssh private key) and known_hosts (so that ssh doesn't prompt about the server's fingerprint the first time we upload something).

Here we go, working Instantbird linux build slaves! Figuring out all these details for our first CentOS6 slave took me a few evenings, but doing it again on the second slave was really easy.

Tuesday, May 20 2014

Summer of Code 2014 projects

The coding period is starting for Google Summer of Code 2014. I am pleased to list the 28 projects (7 more than last year!) that are mentored this year under the Mozilla banner.

The title of each project is linked to the location where the student will be posting weekly updates on their progress, if you want to follow along with a project you are interested in. I’m sure they would appreciate any help or advice you have :-) Please help them write useful code during the summer!

Project Title Student Mentor
MDN – Bug Fixing and Polishing Akshay Aurora David Walsh
Firefox OS Games Resources Andre Alves Garzia Soledad Penades
Development of both online as offline speech recognition to B2G and Firefox Andre Natal Guilherme Gonçalves
Porting key Meemoo modules to NoFlo Vilson Vieira Forrest Oliphant
Functional Test Suite and Features for QA Taskboard - One and Done Pankaj Malhotra Bob Silverberg
Implement Zest recorder and runner Sunny Gogoi Simon Bennetts
Improve font handling and text shaping in Servo Patryk Obara Patrick Walton
Add learning capability in the Gaia Keyboard prediction Sukant Garg Jan Jongboom
Browsercast Gabriel Ivanica Greg Wilson
Migrate from YUI2 to YUI3 . Bug 453268 Ishitva Goel Byron Jones
Implement Speech Synthesis on Desktop Firefox Yashasvi Girdhar Eitan
Add markdown support to Bugzilla Koosha Khajehmoogahi David Lawrence
Improved messaging functionality and interaction in the SMS app Kumar Rishav Gabriele Svelto
Update Lightning Invitations to Latest Specification Malintha Fernando Mohit Kanwal
Implement XMLHttpRequest in Servo Manish Goregaokar Josh Matthews
Instantbird: WebRTC Support Mayank Kumar BenediktP
Instantbird: Indexed Chat History and Infinite Conversation Scrollback Nihanth Subramanya aleth
Peer instruction on the Web Piotr Banaszkiewicz mhoye
Implementation of SVG Backend for PDF.js Pramodh KP yury
A Desktop Environment based on B2G Towfique Anam Fabrice Desré
Math virtual keyboard Raniere Silva Salva
Improve Calendar Backends Reid Anderson Philipp Kewisch
Kitherder, web application for the Security Mentorships program Renee Cheung Yvan Boily
FileLinks in IMs / File transfer Saurabh Anand clokep
Thunderbird - Make the unit test framework work with maildir mailbox format Suyash Agarwal R Kent James
Mozilla Intellego -- Terminology-driven automatic translation of web sites. Tharshan Muthulingam Jeff
Implement Electrolysis Support for Addon SDK Tomislav Jovanovic gozala
Mochitest Failure Investigator Vaibhav Agrawal joelmaher

Tuesday, April 22 2014

Thoughts about mentoring Summer of Code students

I've mentored Google Summer of Code students 3 years in a row, and I would like to share with new GSoC mentors what I've learned, in the hope that it can help them get more out of the summer. I'm going to assume that most mentors share these 3 goals:
  • get usable code at the end of the summer
  • train long term contributors who will stay active after the end of the summer
  • avoid spending disproportionate amounts of time mentoring.
I can't overstate how important it is to give the student good working habits from the very beginning. If you setup a good working relationship from the beginning, you are on for an awesome summer. If you let a student do things that frustrate you without sending immediate feedback about it, you'll both suffer during the whole summer.
Here are some frequent problems that are likely to happen if you don't pay enough attention in the beginning:
  • Because of the way school works, students are used to hiding their work until it's ready to be evaluated. Training them to show work-in-progress code so that you can provide early feedback and effectively help them takes time. You'll likely need to ask "show me the code, please" several times, encourage them, and comfort them that it's OK to show unfinished non-working code.
  • Students are used to receiving feedback from one person, and are likely to send their work only to their mentor when requesting feedback or expecting evaluation. Encourage the student to discuss the project with all the relevant people and more generally to interact with the community; avoid private discussion of technical matters. Doing this has 2 interesting consequences: the time you'll spend mentoring the student will be dramatically reduced because the mentoring load will be spread across all your team (for me this often meant that when my student asked a question on IRC, he received a good and timely answer before I had even read the question!), and the student will feel like he's part of your team, which increases the likelihood of staying around after the summer.
  • Unless given specific directions, students tend to work on a branch or on a repository somewhere, and think about merging their code or getting a review only at the end of the summer. It's common to see student applications where the schedule says that they will do all the coding work, and then get a review during the last week of the summer to land the code. Experienced Mozilla developers knows there's no way sending a several-months-of-work-worth patch to someone's review queue and expecting a timely review will work. But students usually dramatically underestimate both the time it takes to get a review AND the time it takes to address the feedback (probably because at school they hand their work and only expect a grade, or a pass/fail in return). As a mentor it's your job to ensure they get their code through review. Urge students to write small patches, and get them reviewed as soon as possible. Students (and other developers ;)) will be tempted to 'just add one more little feature before attaching the patch'. Resist! Filing a new bug is easy. If the schedule of a student involves writing code without submitting it for review and inclusion, this raises a red flag for me: is there really no way to break this down to smaller pieces that could land separately? Don't hesitate to rediscuss the schedule with the student if you aren't fully happy with it.
  • Given that the students are being paid money to participate in GSoC, it makes sense that lots of students will consider summer of code as "work", and have money be the primary motivator; which means that once they get the final payment, they disappear. This feeling can't be avoided in all cases, but if we can get the student to instead feel that the money is an opportunity to not do any other work during that time to focus full-time on volunteering for a project they care about, the student is likely to stay around for much longer than the summer. This can be done by treating the student more like a volunteer than like someone paid to be on your team. Get the student to have code landing early, and often. Get the student addicted to restarting his nightly build and seeing his code is in use there. Encourage the student to not only fix things directly related to his GSoC project, but to also spend a few hours here and there fixing his own itches and looking into other bugs that make the product better for his own use cases.
Here is one suggestion to get students started: during the community bounding period, assign them a 'small' project to warm up. The ideal characteristics of such a project are:
  • will get the student to go through all the processes of your team (using bugzilla, attaching patches, requesting review, handling review comments, …)
  • will get the student to touch code areas that will be relevant during the summer.
  • can be completed by the student before the ends of the community bounding period (think of it as a 2-weeks project, but size it like something you would do in a rainy week-end. Remember that most of the time spent by the student will not be actual coding, but learning to interact with your team, and this is the very reason why this project is done during the community bounding period).
  • once this small project lands, it will be highly visible, and will cause the student to receive appreciation from other people in your team/community.
It's not always possible to find a project fitting perfectly here, but it's usually possible to find something close enough to fit the purpose of getting the student started. The Instantbird team will be using this 'small community bounding project' approach to starting the mentoring relationship for the third time this year.
I tend to think that getting the student started on the right track is the most important thing a mentor needs to do, and if it's done correctly the rest of the mentoring is easy and pleasurable.

Thursday, March 20 2014

Summer of Code Student application deadline approaching

This is just a reminder to students interested in applying for Google Summer of Code 2014: the application deadline is "21 March 19:00 UTC" and no late application will be accepted.

If you intend to apply to Google Summer of Code this year and haven't submitted your application yet, don't wait: apply now!

Friday, February 28 2014

Google Summer of Code Student who went on to become a Mentor/Org Admin

Since 2006, I've probably tried every possible role as a Google Summer of Code participant: accepted student, rejected student, submitter of a rejected organization application, mentor of successful students, mentor of a failing student, co-administrator, and now administrator. Here's my story:

  • In 2006, I had the pleasure of being selected to participate as a student for the Mozilla organization. The work I did on the Page Info dialog eventually shipped as part of Firefox 3.0.
  • In 2007 I applied as a student again, to a different organization, but unfortunately wasn't selected. However, I received an email from someone from the organization who told me the application was good and they would have liked to take me as a student if they had received more slots. He offered to mentor me if I decided to move forward with the project anyway, which I did! I completed the project and in October 2007, Instantbird 0.1 was released. Instantbird is a cross platform, easy to use, instant messaging client based on Mozilla technologies.
  • In 2007, I was selected by Mozilla for an internship in the Firefox team, and spent the end of the year in their headquarters in Mountain View, California. I think this was in large part because people were happy about the work I did in 2006 as a Summer of Code student.
  • In 2008, 2009 and 2010, I focused on Instantbird, which had a growing community of volunteers improving it every day. I sent Summer of Code organization applications on behalf of the Instantbird community, but they weren't accepted. When I asked for feedback, I was told that we should try to work with Mozilla.
  • And this is what we did in 2011! Gerv agreed to let us add Instantbird project ideas to the Mozilla idea list. In 2011 I mentored a student, who completed successfully his work on implementing XMPP in JavaScript for Instantbird. In October I attended the mentor summit in Mountain View.
  • In 2012, I mentored another student, who did some excellent work on improving the user experience for new users during the first run of Instantbird.
  • In 2013, Gerv, who had been handling Summer of Code for Mozilla since the beginning in 2005, asked me if I would be interested in eventually replacing him as an Administrator. After some discussion, I accepted the offer, and we agreed to have a transition period. In 2013, I was backup administrator. I also mentored for the third year in a row. Unfortunately I had to fail my student. This was frustrating, but also a learning experience.
  • In 2014, I submitted the Summer of Code organization application on behalf of Mozilla, it was accepted and we are looking forward to another great summer of code!
  • On a more personal note, after being a Mozilla volunteer for years (since 2004), working on the Thunderbird team (from 2011 to 2012), and then on WebRTC apps (since October 2012), I'm starting in March 2014 as a full time engineer in the Firefox team. Which is the team for which I was a Summer of Code student, 8 years ago.

Thursday, February 6 2014

Mozilla and Google Summer of Code, FOSDEM talk

Last Saturday at FOSDEM, I had the pleasure of giving a talk with Gerv about Mozilla's participation in Google Summer of Code. If you missed us, you can see the slides.

This post is also a good opportunity to remind you to add your ideas to our brainstorm wiki page now if you are interested in mentoring a student this summer.

Friday, January 24 2014

Project ideas for Summer of Code 2014

Google will be running this year the 10th edition of Summer of Code. Mozilla has had the pleasure of participating every year so far, and we are hoping to participate again this year. In the next 3 weeks, we need to prepare a list of suitable projects to support our application.

Can you think of a 3-month coding project you would love to guide a student through? This is your chance to get a student focusing on it for 3 months! Summer of Code is a great opportunity to introduce new people to your team and have them work on projects you care about but that aren't on the critical path to shipping your next release.

Here are the conditions for the projects:

  • completing the project should take roughly 3 months of effort;
  • any part of the Mozilla project (Firefox, Firefox OS, Thunderbird, Instantbird, SeaMonkey, Bugzilla, L10n, NSS, IT, and many more) can submit ideas, as long as they require coding work;
  • there is a clearly identified mentor who can guide the student through the project.


If you have an idea, please put it on the Brainstorming page, which is our idea development scratchpad. Please read the instructions at the top – following them vastly increases your chances of your idea getting added to the formal Ideas page.

The deadline to submit project ideas and help us be selected by Google is February 14th.

Please feel free to discuss with me any Summer of Code question you may have.

Tuesday, January 21 2014

Instantbird bugs should now go to bugzilla.mozilla.org

From 2007 to late 2013, Instantbird bugs were tracked on bugzilla.instantbird.org. This was fine as long as Instantbird was completely separate from all other Mozilla products, but when Instantbird's chat backend was re-used to add instant messaging to Thunderbird, we started having bugs that were relevant for both Instantbird and Thunderbird. Having them in two separate bug trackers was messy and caused lots of duplicates.

We started discussing this at MozCampEU2012 in Warsaw. It quickly became apparent that getting a product created for Instantbird and another one for the chat core wasn't a problem, but that importing our existing bugs would require more work. The plan at this time was to import the existing BIO (bugzilla.instantbird.org) database into BMO (bugzilla.mozilla.org). Unfortunately, while it seems bugzilla already supports importing bugs from plenty of other bug trackers, it doesn't support importing bugs from itself. Some people from the BMO team thought writing such an importer would be a nice and not-very-difficult project, so they agreed to work on it. We filed a bug to figure out the details. And nothing happened for months, because there were (understandably) higher priorities for the BMO team. Ten months later, the BMO team concluded that they were unlikely to ever get time for this project, and suggested we work on another solution instead: using BzAPI to do the import ourselves. This was more work than we could get done immediately, so we kept that in mind for later. Patrick Cloke started experimenting with BzAPI but couldn't get enough of my and others' attention to complete the project.

In December 2013, we decided to move forward with the import, and requested the creation of the products just before the holidays. We drafted a plan in which we broke down the work that needed to happen in several steps that could each be performed by a (relatively) simple and independent script:

  1. first, download all the data from bugzilla.mozilla.org in JSON, create a file for the data of each bug
  2. slice the content of each file from 1. into a stream of events (each event is an action performed by the bugzilla users).
  3. transform the sliced events into events that would apply on BMO (eg. change product names when they don't match exactly on BIO and BMO)
  4. merge the events of all bugs into a single file; without duplicates (eg. when marking a bug as a duplicate of another bug, this creates events on both bugs, but we only need to import one, the other happens automatically).
  5. replay events on BMO: this needs to create a map of new bug and attachment numbers, and replace them automatically in comments.

The plan looked simple on paper (well, on an etherpad). Reality got in the way, with plenty of not-really-expected complications:

  • BzAPI has lots of documented (eg. can't create an attachment at the same time as filing a bug; can't set the component without also setting the product, ...) and undocumented (eg. UTF8 is supported when filing a bug or adding comments, but not when commenting on an attachment) limitations.
  • The format of data downloaded from BzAPI is significantly different from the format of data we need to send it to do changes. This is especially true for attachment flags.
  • Sometimes the values in bug fields don't match what's in the history of events that happened to the bug. This problem occurs when users changed their bugzilla email address, when products got renamed. The worst case we had to deal with was a target milestone that was renamed, then a new target milestone was created with the original name, and some bug's milestone was changed from the renamed milestone to the new milestone with the original name. This caused an interesting complication because we ended up with changes where the mapped value was the same before and after the change.
  • We initially started using the js shell for our scripts, but to do HTTP requests, we needed XPCOM, so we switched to xpcshell instead. Unfortunately, xpcshell's readline() function truncates input data after 255 characters. Later I discovered that xpcshell doesn't handle UTF8 input/output by default, and had to use nsIScriptableUnicodeConverter for that.

Our scripts are all in the bio-merge hg repository. The result of step 4. is a 120MB text file with each line being the JSON representation of an event that needs to be replayed. We tested replaying events on bugzilla-dev.allizom.org first for a single bug, then for a few set of bugs that had dependencies. Finally we replayed our whole set of bugs (2294 bugs, 20984 events), and after fixing the issues that surfaced during that test run, we were ready to do the final import.

The final import took 848 minutes to complete (14 hours). 240 events failed to replay because some components didn't have on BMO the exact names the script expected. I fixed the script and replayed these again.

We finished by adding comments in all existing BIO bugs giving the new URL, and closed BIO (it's now read-only).  As of 2013-12-30, new Instantbird bugs should be filed on bugzilla.mozilla.org, in the Instantbird product for UI bugs, in the Chat Core product for backend issue (this part of the code is shared with Thunderbird), and in the Instantbird Servers product of issues with our servers.

Tuesday, June 28 2011

Instantbird 1.0, sorti simultanément en 11 langues (dont le français évidemment)

Instantbird 1.0 vient de sortir.

Instantbird logo

Cette version apporte de très nombreuses améliorations. Cette fois ci, non seulement le logiciel en lui-même est traduit, mais le site web aussi, alors pour savoir ce qui est changé depuis la version précédente, il suffit donc tout simplement d'aller voir sur le site d'Instantbird lui-même :).

Instantbird 1.0 - copie d'écran sur Windows 7

De nombreux blogs reprennent déjà l'information. N'hésitez pas à faire de même (ou à partager sur Facebook ou twitter) afin que vos amis en profitent aussi !

- page 1 of 4