Florian Quèze

To content | To menu | To search


Posts related to Mozilla activities

Entries feed - Comments feed

Monday, February 8 2016

Project ideas wanted for Summer of Code 2016

Google is running Summer of Code again in 2016. Mozilla has had the pleasure of participating many years so far, and even though we weren't selected last year, we are hoping to participate again this year. In the next few weeks, we need to prepare a list of suitable projects to support our application.

Can you think of a 3-month coding project you would love to guide a student through? This is your chance to get a student focusing on it for 3 months! Summer of Code is a great opportunity to introduce new people to your team and have them work on projects you care about but that aren't on the critical path to shipping your next release.

Here are the conditions for the projects:

  • completing the project should take roughly 3 months of effort for a student;
  • any part of the Mozilla project (Firefox, Firefox OS, Thunderbird, Instantbird, SeaMonkey, Bugzilla, L10n, NSS, IT, and many more) can submit ideas, as long as they require coding work;
  • there is a clearly identified mentor who can guide the student through the project.

If you have an idea, please put it on the Brainstorming page, which is our idea development scratchpad. Please follow the instructions at the top.

The deadline to submit project ideas and help us be selected by Google is February 19th.

Note for students: the student application period starts on March 14th, but the sooner you start discussing project ideas with potential mentors, the better.

Tuesday, November 3 2015

What about search hijacking?

Some people may have noticed that lately my work has been concentrated on bugs with 'hijacking' in the status whiteboard, but we never really took the time to explain what the goals of this work are, or aren't.

What we call "search hijacking" is when a user's default search engine is changed without the user's intent, typically while the user is installing some other software on his system. Search hijacking is a very common phenomenon, and is causing a severely degraded user experience for the affected users; sometimes leading users to switch to another browser to stop searching with a search engine they never wanted and don't know how to change.

When the search service (the back-end code handling search engines) was initially created about a decade ago, search hijacking wasn't a thing. Things were designed for maximum user customizability. Unfortunately, these days the ease of customization combined with the monetary incentives offered has made search a hijacking target.

The current situation is untenable for Firefox users, so we had to do something. Deciding what to do wasn't obvious, because the difference between the user customizing something and unwanted software writing things to disk isn't always as clear as we would like it to be.

Whatever we do, hijackers will be able to change their code to workaround our changes. In some way, this is an arms race. We can make changes every 6 weeks if needed. We have reasons to believe that it takes most hijackers several months to update their hacks and deploy them, as they need to be bundled with other software for downloads. Our first approach to the problem, especially while our resources to tackle this were very limited, was to do tiny changes every once in a while to breaks some hijacks and help our users for a few months. We didn't want to do pointless changes, but we could find a few things that didn't feel right in our code, and fixed them.

For example, we had some evidence that a significant number of users had modified yahoo.xml files in their application folder. Being able to hijack the default engine by just changing an URL in a plain XML file on the disk was just ridiculous, especially when almost every other non-binary file used by the browser is packed inside the omni.ja file. We fixed this in bug 1162569 for Firefox 40. Similarly, it was possible to change the default engine by dropping in the user's profile a file redefining the same engine. There is no way to end up in this situation 'naturally', because we refuse to install duplicate engines. The logical fix was to ensure that engines in the profile can't override built-in engines. We fixed this in bug 1109354, also for Firefox 40. This was also a good opportunity to add lots of tests about how engines are loaded.

Something seriously limiting our ability to correctly attack the problem was the lack of good data. We found the evidence of modified yahoo.xml files in the wild almost by accident, while looking at incoherent default engine data we received in FHR. In parallel to the fixes mentioned above, we also introduced a better data collection mechanism, with the addition of a telemetry probe recording details about how the current default engine has been loaded.

I haven't been able to look at release-user data collected through this new telemetry probe yet (I intend to do so soon!), but a quick analysis of the early data we got from nightly and aurora users revealed that a non-trivial amount of users had no engine at all, resulting in search features being completely broken for them. This situation can happen if the hijackers removed all the other 'competing' engines to ensure they are the only one, and then got removed themselves, eg. due to one of our previous fixes. We addressed this problem in bug 438599 for Firefox 43, by unhiding automatically the original default engine when no visible engine is left.

A major change that is coming soon to Firefox is required add-on signing. This is a game changer for hijacking, because now anything that lets malware with disk access insert unsigned-code that changes Firefox's behavior is explicitly a bug. This means we are now in a position to close lots of doors that were wide open, to ensure the control is in the hands of our users, not hijackers. There are 2 supported ways to install new engines: either the user does it through the Firefox UI (eg. using OpenSearch discovery on a website), or it's done by a signed add-on. Any other way left to install a search engine and set it as the default is something we need to look into.

In order to better enforce this, we need to be reasonably confident that the data from the user's profile has actually been written due to intentional user actions. The next step of our hijacking remediation work is to cleanup the way search-related data is stored in user profiles. We used to store 2 JSON files (one cache file dismissed after each update, and one file with metadata), in addition to a plain xml file for each user-installed engine. This made it easy to mess either with the xml files (just dropping in the folder an xml file in the open search format was enough to install an engine) or with the metadata. In bug 1203167 (just landed on nightly), I'm changing the profile storage so that everything is in a single file. The data is stored in a JSON format, but the file is written to disk using OS.File's compression feature, which in addition to reducing the file size on disk will also make the file resistant to dumb search & replace attacks. The important data stored in the file (eg. the name of the user-selected default engine) is protected from tampering using verification hashes. While these hashes are by no way meant to be cryptographically secure, they should be unique to each profile, and so make tampering harder. Changing the storage format was also an opportunity to save slightly more data about how users installed engines. For example, if an engine was installed from OpenSearch discovery, we will remember the hostname of the website from which it was downloaded.

We are not done. We have a few more doors to close. We also don't expect to ever be completely resistant to hijacking, as an attacker with local admin access can do whatever. We mostly aim to make hijacking hard enough that it will require some (minimal) engineering skills to do it. Dropping a file on disk or doing a search & replace on the default engine's URL shouldn't be enough. It should require running local code. We can’t protect users from all their mistakes, but we can draw a line between malware-behavior and user-customization that's clear enough for antivirus vendors to enforce it for their users.

We are still working on preventative measures. But a few releases from now, we can expect to start offering curative measures. With Firefox 45 we'll start recording information about engines origin. At some point (currently targeting Firefox 46), we could offer users who have non-default engines of unknown origin an opportunity to check their search settings and revert to the original default if they don't remember changing their default engine themselves. UX is currently thinking this through.

Tuesday, March 3 2015

Mozilla not accepted for Google Summer of Code 2015

As you may have already seen, Mozilla is not in the list of organizations accepted for Google Summer of Code 2015.

People who have observed the list carefully may have noticed that there are fewer accepted organizations this year: 137 (down from 190 in 2014 and 177 in 2013). Other organizations that have participated successfully several times are also not in the 2015 list (eg. Linux Foundation, Tor, ...).

After a quick email exchange with Google last night, here is the additional information I have:
  • not accepting Mozilla was a difficult decision for them. It is not the result of a mistake on our part or an accident on their side.
  • there's an assumption that not participating for one year would not be as damaging for us as it would be for some other organizations, due to us having already participated many times.
  • this event doesn't affect negatively our chances of being selected next year, and we are encouraged to apply again.

This news has been a surprise for me. I am disappointed, and I'm sure lots of people reading this are disappointed too. I would like to thank all the people who considered participating this year with Mozilla, and especially all the Mozillians who volunteered to mentor and contributed great project ideas. I would also like to remind students that while Summer of Code is a great opportunity to contribute to Mozilla, it's not the only one. Feel free to contact mentors if you would like to work on some of the suggested ideas anyway.

Let's try again next year!

Wednesday, February 11 2015

FOSDEM 2015 slides - What's new in Firefox?

This year at FOSDEM I talked about Firefox. I summarized the major changes that have been made to Firefox in 2014, and what our plans are for 2015.

Here are the slides of my FOSDEM presentation:
What's new in Firefox? What happened in 2014? What's the plan for 2015? FOSDEM 2015, by Florian Quèze

Thursday, January 22 2015

Project ideas wanted for Summer of Code 2015

Google is running Summer of Code again in 2015. Mozilla has had the pleasure of participating every year so far, and we are hoping to participate again this year. In the next few weeks, we need to prepare a list of suitable projects to support our application.

Can you think of a 3-month coding project you would love to guide a student through? This is your chance to get a student focusing on it for 3 months! Summer of Code is a great opportunity to introduce new people to your team and have them work on projects you care about but that aren't on the critical path to shipping your next release.

Here are the conditions for the projects:

  • completing the project should take roughly 3 months of effort for a student;
  • any part of the Mozilla project (Firefox, Firefox OS, Thunderbird, Instantbird, SeaMonkey, Bugzilla, L10n, NSS, IT, and many more) can submit ideas, as long as they require coding work;
  • there is a clearly identified mentor who can guide the student through the project.

If you have an idea, please put it on the Brainstorming page, which is our idea development scratchpad. Please read the instructions at the top – following them vastly increases chances of your idea getting added to the formal Ideas page.

The deadline to submit project ideas and help us be selected by Google is February 20th.

Note for students: the student application period starts on March 16th, but the sooner you start discussing project ideas with potential mentors, the better.

Please feel free to discuss with me any question you may have related to Mozilla's participation in Summer of Code. Generic Summer of Code questions are likely already answered in the FAQ.

Sunday, July 27 2014

Firefox is awesome

"Hey, you know what? Firefox is awesome!" someone exclaimed at the other end of the coworking space a few weeks ago. This piqued my curiosity, and I moved closer to see what she was so excited about.

When I saw the feature she was delighted of finding, it reminded me a similar situation several years ago. In high school, I was trying to convince a friend to switch from IE to Mozilla. The arguments about respecting web standards didn't convince him. He tried Mozilla anyway to please me, and found one feature that excited him.
He had been trying to save some images from webpages, and for some reason it was difficult to do (possibly because of context menu hijacking, which was common at the time, or maybe because the images were displayed as a background, …). He had even written some Visual Basic code to parse the saved HTML source code and find the image urls, and then downloaded them, but the results weren't entirely satisfying.
Now with Mozilla, he could just right click, select "View Page Info", click on the "Media" tab, and find a list of all the images of the page. I remember how excited he looked for one second, until he clicked a background image in the list and the preview stayed blank; he then clicked the "Save as" button anyway and… nothing happened. Turns out that "Save as" button was just producing an error in the Error Console. He then looked at me, very disappointed, and said that my Mozilla isn't ready yet.
After that disappointment, I didn't insist much on him using Mozilla instead of IE (I think he did switch anyway a few weeks or months later).

A few months later, as I had time during summer vacations, I tried to create an add-on for the last thing I could do with IE but not Firefox: list the hostnames that the browser connects to when loading a page (the add-on, View Dependencies, is on AMO). I used this to maintain a hosts file that was blocking ads on the network's gateway.
Working on this add-on project caused me to look at the existing Page Info code to find ideas about how to look through the resources loaded by the page. While doing this, I stumbled on the problem that was causing background image previews to not be displayed. Exactly 10 years ago, I created a patch, created a bugzilla account (I had been lurking on bugzilla for a while already, but without creating an account as I didn't feel I should have one until I had something to contribute), and attached the patch to the existing bug about this background preview issue.
Two days later, the patch was reviewed (thanks db48x!), I addressed the review comment, attached a new patch, and it was checked-in.
I remember how excited I was to verify the next day that the bug was gone in the next nightly, and how I checked that the code in the new nightly was actually using my patch.

A couple months later, I fixed the "Save as" button too in time for Firefox 1.0.

Back to 2014. The reason why someone in my coworking space was finding Firefox so awesome is that "You can click "View Page Info", and then view all the images of the page and save them." Wow. I hadn't heard anybody talking to me about Page Info in years. I did use it a lot several years ago, but don't use it that much these days. I do agree with her that Firefox is awesome, not really because it can save images (although that's a great feature other browsers don't have), but because anybody can make it better for his/her own use, and by doing so making it awesome for millions of other people now but also in the future. Like I did, ten years ago.

Friday, July 25 2014

Converting old Mac minis into CentOS Instantbird build slaves

A while ago, I received a few retired Mozilla minis. Today 2 of them started their new life as CentOS6 build slaves for Instantbird, which means we now have Linux nightlies again! Our previous Linux build slave, running CentOS5, was no longer able to build nightlies based on the current mozilla-central code, and this is the reason why we haven't had Linux nightlies since March. We know it's been a long wait, but to help our dear Linux testers forgive us, we started offering 64bit nightly builds!

For the curious, and for future reference, here are the steps I followed to install these two new build slaves:

Partition table

The Mac minis came with a GPT partition table and an hfs+ partition that we don't want. While the CentOS installer was able to detect them, the grub it installed there didn't work. The solution was to convert the GPT partition table to the MBR older format. To do this, boot into a modern linux distribution (I used an Ubuntu 13.10 live dvd that I had around), install gdisk (sudo apt-get update && sudo apt-get gdisk) and use it to edit the disk's partition table:

sudo gdisk /dev/sda
Press 'r' to start recovery/transformation, 'g' to convert from GPT to MBR, 'p' to see the resulting partition table, and finally 'w' to write the changes to disk (instructions initially from here).
Exit gdisk.
Now you can check the current partition table using gparted. At this point I deleted the hfs+ partition.

Installing CentOS

The version of CentOS needed to use the current Mozilla build tools is CentOS 6.2. We tried before using another (slightly newer) version, and we never got it to work.

Reboot on a CentOS 6.2 livecd (press the 'c' key at startup to force the mac mini to look for a bootable CD).
Follow the instructions to install CentOS on the hard disk.
I customized the partition table a bit (50000MB for /, 2048MB of swap space, and the rest of the disk for /home).

The only non-obvious part of the CentOS install is that the boot loaded needs to be installed on the MBR rather than on the partition where the system is installed. When the installer asks where grub should be installed, set it to /dev/sda (the default is /dev/sha2, and that won't boot). Of course I got this wrong in my first attempts.

Installing Mozilla build dependencies

First, install an editor that is usable to you. I typically use emacs, so: sudo yum install emacs

The Mozilla Linux build slaves use a specifically tweaked version of gcc so that the produced binaries have low runtime dependencies, but the compiler still has the build time feature set of gcc 4.7. If you want to use something as old as CentOS6.2 to build, you need this specific compiler.

The good thing is, there's a yum repository publicly available where all the customized mozilla packages are available. To install it, create a file named /etc/yum.repos.d/mozilla.repo and make it contain this:


Adapt the baseurl to finish with i386 or x86_64 depending on whether you are making a 32 bit or 64 bit slave.

After saving this file, you can check that it had the intended effect by running this comment to list the packages from the mozilla repository: repoquery -q --repoid=mozilla -a

You want to install the version of gcc473 and the version of mozilla-python27 that appear in that list.

You also need several other build dependencies. MDN has a page listing them:

yum groupinstall 'Development Tools' 'Development Libraries' 'GNOME Software Development'
yum install mercurial autoconf213 glibc-static libstdc++-static yasm wireless-tools-devel mesa-libGL-devel alsa-lib-devel libXt-devel gstreamer-devel gstreamer-plugins-base-devel pulseaudio-libs-devel

Unfortunately, two dependencies were missing on that list (I've now fixed the page):
yum install gtk2-devel dbus-glib-devel

At this point, the machine should be ready to build Firefox.

Instantbird, because of libpurple, depends on a few more packages:
yum install avahi-glib-devel krb5-devel

And it will be useful to have ccache:
yum install ccache

Installing the buildbot slave

First, install the buildslave command, which unfortunately doesn't come as a yum package, so you need to install easy_install first:

yum install python-setuptools python-devel mpfr
easy_install buildbot-slave

python-devel and mpfr here are build time dependencies of the buildbot-slave package, and not having them installed will cause compiling errors while attempting to install buildbot-slave.

We are now ready to actually install the buildbot slave. First let's create a new user for buildbot:

adduser buildbot
su buildbot
cd /home/buildbot

Then the command to create the local slave is:

buildslave create-slave --umask=022 /home/buildbot/buildslave buildbot.instantbird.org:9989 linux-sN password

The buildbot slave will be significantly more useful if it starts automatically when the OS starts, so let's edit the crontab (crontab -e) to add this entry:
@reboot PATH=/usr/local/bin:/usr/bin:/bin /usr/bin/buildslave start /home/buildbot/buildslave

The reason why the PATH environment variable has to be set here is that the default path doesn't contain /usr/local/bin, but that's where the mozilla-python27 packages installs python2.7 (which is required by mach during builds).

One step in the Instantbird builds configured on our buildbot use hg clean --all and this requires the purge mercurial extension to be enabled, so let's edit ~buidlbot/.hgrc to look like this:
$ cat ~/.hgrc
purge =

Finally, ssh needs to be configured so that successful builds can be uploaded automatically. Copy and adapt ~buildbot/.ssh from an existing working build slave. The files that are needed are id_dsa (the ssh private key) and known_hosts (so that ssh doesn't prompt about the server's fingerprint the first time we upload something).

Here we go, working Instantbird linux build slaves! Figuring out all these details for our first CentOS6 slave took me a few evenings, but doing it again on the second slave was really easy.

Tuesday, May 20 2014

Summer of Code 2014 projects

The coding period is starting for Google Summer of Code 2014. I am pleased to list the 28 projects (7 more than last year!) that are mentored this year under the Mozilla banner.

The title of each project is linked to the location where the student will be posting weekly updates on their progress, if you want to follow along with a project you are interested in. I’m sure they would appreciate any help or advice you have :-) Please help them write useful code during the summer!

Project Title Student Mentor
MDN – Bug Fixing and Polishing Akshay Aurora David Walsh
Firefox OS Games Resources Andre Alves Garzia Soledad Penades
Development of both online as offline speech recognition to B2G and Firefox Andre Natal Guilherme Gonçalves
Porting key Meemoo modules to NoFlo Vilson Vieira Forrest Oliphant
Functional Test Suite and Features for QA Taskboard - One and Done Pankaj Malhotra Bob Silverberg
Implement Zest recorder and runner Sunny Gogoi Simon Bennetts
Improve font handling and text shaping in Servo Patryk Obara Patrick Walton
Add learning capability in the Gaia Keyboard prediction Sukant Garg Jan Jongboom
Browsercast Gabriel Ivanica Greg Wilson
Migrate from YUI2 to YUI3 . Bug 453268 Ishitva Goel Byron Jones
Implement Speech Synthesis on Desktop Firefox Yashasvi Girdhar Eitan
Add markdown support to Bugzilla Koosha Khajehmoogahi David Lawrence
Improved messaging functionality and interaction in the SMS app Kumar Rishav Gabriele Svelto
Update Lightning Invitations to Latest Specification Malintha Fernando Mohit Kanwal
Implement XMLHttpRequest in Servo Manish Goregaokar Josh Matthews
Instantbird: WebRTC Support Mayank Kumar BenediktP
Instantbird: Indexed Chat History and Infinite Conversation Scrollback Nihanth Subramanya aleth
Peer instruction on the Web Piotr Banaszkiewicz mhoye
Implementation of SVG Backend for PDF.js Pramodh KP yury
A Desktop Environment based on B2G Towfique Anam Fabrice Desré
Math virtual keyboard Raniere Silva Salva
Improve Calendar Backends Reid Anderson Philipp Kewisch
Kitherder, web application for the Security Mentorships program Renee Cheung Yvan Boily
FileLinks in IMs / File transfer Saurabh Anand clokep
Thunderbird - Make the unit test framework work with maildir mailbox format Suyash Agarwal R Kent James
Mozilla Intellego -- Terminology-driven automatic translation of web sites. Tharshan Muthulingam Jeff
Implement Electrolysis Support for Addon SDK Tomislav Jovanovic gozala
Mochitest Failure Investigator Vaibhav Agrawal joelmaher

Tuesday, April 22 2014

Thoughts about mentoring Summer of Code students

I've mentored Google Summer of Code students 3 years in a row, and I would like to share with new GSoC mentors what I've learned, in the hope that it can help them get more out of the summer. I'm going to assume that most mentors share these 3 goals:
  • get usable code at the end of the summer
  • train long term contributors who will stay active after the end of the summer
  • avoid spending disproportionate amounts of time mentoring.
I can't overstate how important it is to give the student good working habits from the very beginning. If you setup a good working relationship from the beginning, you are on for an awesome summer. If you let a student do things that frustrate you without sending immediate feedback about it, you'll both suffer during the whole summer.
Here are some frequent problems that are likely to happen if you don't pay enough attention in the beginning:
  • Because of the way school works, students are used to hiding their work until it's ready to be evaluated. Training them to show work-in-progress code so that you can provide early feedback and effectively help them takes time. You'll likely need to ask "show me the code, please" several times, encourage them, and comfort them that it's OK to show unfinished non-working code.
  • Students are used to receiving feedback from one person, and are likely to send their work only to their mentor when requesting feedback or expecting evaluation. Encourage the student to discuss the project with all the relevant people and more generally to interact with the community; avoid private discussion of technical matters. Doing this has 2 interesting consequences: the time you'll spend mentoring the student will be dramatically reduced because the mentoring load will be spread across all your team (for me this often meant that when my student asked a question on IRC, he received a good and timely answer before I had even read the question!), and the student will feel like he's part of your team, which increases the likelihood of staying around after the summer.
  • Unless given specific directions, students tend to work on a branch or on a repository somewhere, and think about merging their code or getting a review only at the end of the summer. It's common to see student applications where the schedule says that they will do all the coding work, and then get a review during the last week of the summer to land the code. Experienced Mozilla developers knows there's no way sending a several-months-of-work-worth patch to someone's review queue and expecting a timely review will work. But students usually dramatically underestimate both the time it takes to get a review AND the time it takes to address the feedback (probably because at school they hand their work and only expect a grade, or a pass/fail in return). As a mentor it's your job to ensure they get their code through review. Urge students to write small patches, and get them reviewed as soon as possible. Students (and other developers ;)) will be tempted to 'just add one more little feature before attaching the patch'. Resist! Filing a new bug is easy. If the schedule of a student involves writing code without submitting it for review and inclusion, this raises a red flag for me: is there really no way to break this down to smaller pieces that could land separately? Don't hesitate to rediscuss the schedule with the student if you aren't fully happy with it.
  • Given that the students are being paid money to participate in GSoC, it makes sense that lots of students will consider summer of code as "work", and have money be the primary motivator; which means that once they get the final payment, they disappear. This feeling can't be avoided in all cases, but if we can get the student to instead feel that the money is an opportunity to not do any other work during that time to focus full-time on volunteering for a project they care about, the student is likely to stay around for much longer than the summer. This can be done by treating the student more like a volunteer than like someone paid to be on your team. Get the student to have code landing early, and often. Get the student addicted to restarting his nightly build and seeing his code is in use there. Encourage the student to not only fix things directly related to his GSoC project, but to also spend a few hours here and there fixing his own itches and looking into other bugs that make the product better for his own use cases.
Here is one suggestion to get students started: during the community bounding period, assign them a 'small' project to warm up. The ideal characteristics of such a project are:
  • will get the student to go through all the processes of your team (using bugzilla, attaching patches, requesting review, handling review comments, …)
  • will get the student to touch code areas that will be relevant during the summer.
  • can be completed by the student before the ends of the community bounding period (think of it as a 2-weeks project, but size it like something you would do in a rainy week-end. Remember that most of the time spent by the student will not be actual coding, but learning to interact with your team, and this is the very reason why this project is done during the community bounding period).
  • once this small project lands, it will be highly visible, and will cause the student to receive appreciation from other people in your team/community.
It's not always possible to find a project fitting perfectly here, but it's usually possible to find something close enough to fit the purpose of getting the student started. The Instantbird team will be using this 'small community bounding project' approach to starting the mentoring relationship for the third time this year.
I tend to think that getting the student started on the right track is the most important thing a mentor needs to do, and if it's done correctly the rest of the mentoring is easy and pleasurable.

Thursday, March 20 2014

Summer of Code Student application deadline approaching

This is just a reminder to students interested in applying for Google Summer of Code 2014: the application deadline is "21 March 19:00 UTC" and no late application will be accepted.

If you intend to apply to Google Summer of Code this year and haven't submitted your application yet, don't wait: apply now!

- page 1 of 5