Log in

No account? Create an account
Previous Entry Share Next Entry
Panning in Mozilla Fennec and IFRAMEs
On the desktop, scrolling a page involves changing the page's scroll position.  This uses OS-widgets like scrollbars -- the OS will copy part of the page to the new position on screen and ask the browser to draw the part that's exposed.

On Fennec, we instead have an off-screen browser instance.  We ask it to render parts of the page into a large canvas element that we reposition as the user drags.  This canvas is larger than the visible area, so usually a pan won't show areas beyond the canvas.  When the pan is over, we reposition things and redraw so the next pan can proceed as normal.

This has been the source of a lot of bugs.  It's been fairly easy through kinetic scrolling and zooming to get the browser into a situation where it didn't fix up the canvas properly or start drawing again, letting the checkerboard or gray background show through.  In severe cases, you can get the whole screen to be checkerboard, which effectively hangs the browser as you can't pan back to any of the controls.

Recently, one of our summer interns, Roy, has been working with Stuart on a new system to deal with these problems called the Tile Cache.  Rather than have one giant canvas, we now have a lot of smaller canvases that we draw into as needed.  Tiles can be moved around by the cache so there's always a cloud of them around the ones visible on screen.  It seems to make panning a lot faster, but it's required a lot of changes to input handling code and user interface code, so it's not yet landed in our main Fennec development trunk.

While this has been happening, I've been working on a patch to support panning of IFRAME and FRAMESET content.  Unfortunately, IFRAME panning is going to be a bit slower than page panning, as we effectively just change the scroll position of the embedded frame, which then causes a lot of the page to be invalidated and redrawn.  There are some more complicated techniques that could be explorer for avoiding the whole-area invalidation, but they probably won't be done for the 1.0 release.

To get IFRAME panning to work, when you start a drag, we have to dig into the page content and see if your finger has touched a frame element on the screen.  If so, we scroll that element based on the finger movement.  If you get to the edge, we then move its parent; this is needed to prevent you getting into a situation where you can't pan over to the controls because you zoomed into a IFRAME's area.  One popular site where the whole page is an IFRAME is Google Mail, so without the overflow movement, it would be easy to get stuck.

With luck and a lot of code reviews, all of this should land in time for the third beta of Fennec for the Maemo platform.  I think it wll be a much better Fennec experience, and with IFRAME scrolling, we'll do something the other mobile browsers like Safari on the iPhone don't support.

  • 1
This reminds me of a conversation I had with some of my superiors at Motorola a few months before my project was canceled.

I had been ranting to them that a team of our size and competence needed to be defining our own platform instead of letting Microsoft dictate our platform to us. They told me I was nuts, but my response was something like this -- "No. Think about it. Linux works on an ARM just fine. So, there's our OS and HAL. Next, we license out a widget set so that we have the basic tools to build menus and the like. But we only use this for a minimum of application chrome. Then we port over the one app that we need, which is Mozilla. Mozilla will be our desktop, our apps launcher, and our everything. Our apps will mostly be rich AJAX applications, with extensions to the Javascript as needed to empower our technology. This is a very clear path to realizing a platform quickly."

One of them quipped "Port Mozilla to mobile devices. Good luck with that one."

Seems I might have had a touch of the hubris about doing that.

I think that could work... one of the big problems of Fennec is having to support any random web page. If you're more constrained to smaller widgets, a lot of the performance problems we've been working on solving aren't as much of an issue. Still, it's taking a lot of effort to get good performance on the mobile hardware; the lack of hardware acceleration for graphics is one problem, but the lack of memory bandwidth and CPU time is the real obstacle. Plus lots of random things like the really poor performance of WinMo's file system layer. It's been a challenge.

Oh, the fun I could have telling you the gremlins I've seen in WM's file system layer.

And while I do think that a more constrained set of "pages" for local desktop apps would be fine...there's still the challenge that it's also the browser and thus needs to support any arbitrary web page, too. I mean, I guess we could have beat Mozilla into the "browser" for our desktop and local stuff and licensed Opera to handle the web browsing. That's a funny thought.

Why does Fennec implement panning/scrolling in a different way from the browser?

The tiled approach will probably help the amount of redraw of the content from the engine. But would it also make sense to use HW engine, such as GPU, to provide the smooth scrolling/panning if the offset is small?

Many of the devices we're targeting don't have hardware acceleration. On Windows Mobile, a lot of the screen drivers can do basic blitting, but not much else.

The initial version of Fennec did scroll the page the same was as Firefox does, but it ended up being rather slow. The tile approach is allowing us to do screen-to-screen blitting when possible, only redrawing the portion of the tiles that are newly exposed.

  • 1