! Please note that this is a snapshot of our old Bugzilla server, which is read only since May 29, 2020. Please go to gitlab.xfce.org for our new server !
Make xfce4-session start faster by parallellizing subproces startup
Status:
RESOLVED: FIXED
Severity:
enhancement
Product:
Xfce4-session
Component:
General

Comments

Description Auke Kok editbugs 2008-08-22 20:42:12 CEST
Arjan and me are working on making Xfce4 start up faster. As you know perhaps
we're looking at having Xfce4 natively available or installed in the moblin2
distribution that we are currently working on, and startup time is a hot item.

One of the issues we've found is that xfce4-session waits for each process to
complete forking before starting the next part (xfdesktop, panel, thunar etc...).
This part is easily acceleratable and cuts down startup time quite a bit, from
like 2.5secs to 1.5secs or so, if not more.

Downside of this patch is that the splash screen is possibly off, but we are going
to completely disable the splash screen anyway since splash screens are evil and
waste time. I would even strongly suggest that Xfce as a whole disables the splash
screen by default: it takes longer to actually paint the throbbing mouse (hah!)
than it actually does to start up most of the xfce4 components.

Below patch does 2 things:
- parallellize session component startup
- disable startup notification for the terminal in case it was part of an
xfce4-session. (this is a more optional component but shows that if you exited
with a few terminals open you waste more time repainting the mouse cursor).

please check if the patch parts below are correct and can be perhaps improved on.
Comments welcome as usual :)
Comment 1 Auke Kok editbugs 2008-08-22 20:45:15 CEST
Created attachment 1784 
make subprocess startup async.
Comment 2 Yves-Alexis Perez editbugs 2008-09-09 19:48:45 CEST
Hmhmh, is that really normal to still test startup_done after a while(!startup_done) ?

Cheers :)
Comment 3 Brian J. Tarricone (not reading bugmail) 2008-09-11 09:03:14 CEST
Ok, I'm teaching myself how xfce4-session works.  The patch here is unsurprisingly quite a bit incomplete.  Want to jot down a couple things before I forget.

xfsm_startup_child_watch() will call xfsm_manager_startup_continue() if it seems like an app died and failed to register with the SM.  With parallel startup, this isn't necessary... and is probably also broken since we're looping our asses off in xfsm_manager_startup_continue() running xfsm_startup_continue() already.

As you've noted, this completely breaks the splash screen.  Disabling it might be fine for you guys, but I don't feel comfortable effectively removing it, so we need to "fix" it somehow (Auke, I don't care how slow you think it is; it's not meant to be fast, it's meant to be pretty and informative).

I'm not seeing why disabling startup notification is a particularly good idea either.  It's harmless, and I seriously doubt changing the mouse cursor is all that resource-intensive (it's a hardware cursor for most people anyway, probably).  If it's that much of a big deal on moblin, you'll just have to make sure that none of the .desktop files in autostart have StartupNotify=true in them.  Easy enough.

As Corsac says, no need to recheck startup_done after the while loop.  Minor issue.

I don't really like spinning in that while loop waiting for startup_done.  We don't return control to the glib main loop at all that way, so we'll likely get a backlog of ICE messages while starting apps.  That sounds a bit inefficient to me, and clients might actually block if the ICE socket's buffer fills up (probably would take quite a few clients for that, though it's worthwhile to worry about that case).  Need to think about this.

xfsm_manager_register_client() also calls xfsm_manager_startup_continue(), which is unnecessary in parallel mode.

[ time lapse... ]

Actually, as I look over this code a more and more, it's just not architected correctly for parallel startup.  The patch does the job more or less, but is really just a nasty hack.  I need to figure out a new way of doing the session startup sequence.

The call to xfsm_manager_handle_failed() so soon after startup_done strikes me as incorrect with parallel startup.  I feel like slower clients could get dropped out of the session that way (they'd still run in the current session, I believe, but wouldn't run on subsequent startups).  You might just luck out and all apps register themselves quickly enough, or... you might not.  (Actually, I'm not sure if it's possible for apps to even register at all since we don't give control back to the main loop until after all this crap is done.)

Really xfsm_manager_startup_continue() should just go away and be replaced with xfsm_manager_startup_all(), which goes over pending_properties and g_idle_add()s all of them with an idle function that does more or less what xfsm_startup_continue_session() does.  At the end of each run of that idle function, it should check if pending_properties is empty (which means all session-managed apps have been exec()ed), and then start a timer that, when it times out, will assume any app that hasn't registered yet has failed to do so for some reason and remove them from the session via xfsm_manager_handle_failed() -- need to come up with a good timeout value.  While the timer is counting down (and in the spaces between the idle functions), the main loop is running, and we're processing SM client requests, so everything is moving along.

Ok, that's just a brain dump from an hour and a half of learning how xfce4-session starts itself up.  These notes are mainly for me so I don't forget what I was thinking about, but if anyone else wants to chime in with more suggestions, feel free.
Comment 4 Brian J. Tarricone (not reading bugmail) 2008-09-15 08:38:39 CEST
Created attachment 1817 
first cut at real parallel startup

Ok, kids, here's a first run at parallel startup done the right way.  I just put this together in a couple hours tonight.  Xfce does indeed start up; not sure if it's faster (I just started in Xephyr, with Xfce already running, so...).

I'm also not sure all the semantics are exactly correct, and I'm not even sure that everything that gets started actually registers itself with the session manager properly.

There's also the matter of the registration timeout.  With serial startup, it was "easy" -- you have one app launching at once, and you wait 8 seconds for it to register, and if it doesn't, it failed.  With parallel startup, you more or less have all apps launching at the same time, all vying for the CPU's attention, and later all vying for the session manager's attention.  Will 8 seconds be enough?  Probably not.  I left it at 8 for now -- each app gets its own timer, and the timer doesn't start until after I exec() the client, but...  I think what's probably necessary is some sort of adaptive timeout value that takes into account how many clients are getting started.

Another option would be semi-parallel startup: implement something like a TCP window size for the session manager.  Allow only a certain number of client launches in progress at once, and when a client finishes or fails, start more clients up to that number, and so on.  That might even be faster than starting everything as fast as possible, but it's certainly more complex, and choosing the number for the max number of clients in flight might be difficult.

Anyhow, give this patch a try... but I'd suggest you just run xfce4-session from inside the srcdir... certainly don't overwrite your 'stable' copy.
Comment 5 Brian J. Tarricone (not reading bugmail) 2008-09-15 08:59:58 CEST
(In reply to comment #4)
> I'm also not sure all the semantics are exactly correct, and I'm not even sure
> that everything that gets started actually registers itself with the session
> manager properly.

Ok, well I just tested this, and it appears to work.  Adding stuff to the session, saving session on logout, and then restarting, appears to work.  So that's a good sign.  Still needs lots of testing, tho.
Comment 6 Brian J. Tarricone (not reading bugmail) 2008-09-15 23:10:11 CEST
Random note:
http://foo-projects.org/pipermail/xfce4-dev/2008-September/025077.html

Will have to start things in serial/parallel batches anyway to handle priorities.

Soooo...  The common case is going to be everything having the same priority, so doing batches based solely on prio doesn't solve our timeout issue.

I'm still thinking doing things in smaller parallel batches in general might be a good idea -- sorta like how GThreadPool works -- you set the max threads to, say 5, and then start feeding it jobs; it'll process the job queue in order, and when a thread finishes a job, it'll fetch the next one from the queue, never having more than 5 run at a time.  I think something like this could work well here, just not threaded, of course.  This semi-solves the timeout issue, since there's an upper bound on how much "stuff" is going on at once, and, if I tune it properly, might even be faster than a brute-force start-everything-at-once approach.

I'll try to work on this tonight...
Comment 7 Brian J. Tarricone (not reading bugmail) 2008-09-30 09:34:04 CEST
Created attachment 1849 
parallel startup while respecting priority order

Ok, this guy does parallel startup, but respects the startup priority of each client.  This doesn't limit the number of "in flight" startups, so it's still possible that some clients might fail to register in the required 8 seconds if there are lots of clients in a particular priority group.

Yes, this will likely start up slower than my last patch, but this one is correct.  If you want roughly identical performance to the last patch, you need to set every client's priority to the same value in the session file.

This is tested almost not at all -- I only verified that the session starts up.  Use at your own risk.
Comment 8 Brian J. Tarricone (not reading bugmail) 2008-09-30 09:41:22 CEST
(In reply to comment #7)
> Created an attachment (id=1849) [details]
> parallel startup while respecting priority order

Hmm, this patch is also missing the g_idle_add() stuff for each client that I did for the last one.  I'm not really sure which approach is better, tho...
Comment 9 Stephan Arts editbugs 2008-09-30 09:58:53 CEST
It seems to work here, I'd like to have your patch in beta1, so it gets some coverage from beta-testers.
Comment 10 Brian J. Tarricone (not reading bugmail) 2008-10-01 08:14:20 CEST
Ok, I've committed the patch in attachment #1849  to svn trunk.  It seems to
work decently well.  PLEASE test.
Comment 11 Christoph Wickert editbugs 2008-10-03 12:05:45 CEST
(In reply to comment #10)
> PLEASE test.

Does not work out here (Xfce 4.4.2 on Fedora 9):
With the second patch (attachment #1817 ) I noticed some applcaions starting twice: Once from /etc/xdg/autostart and once from xfce4-session, although I had _not_ selected to save my session and wiped the session cache.

With the parallel startup patch v1 (attachment #1849 ) it's even worse: The startup order seems not predictable, so the panels have a different layout every time I log in. I already had plugins (places, xfce-menu) sitting _in_ the tray instead of the panel!
Comment 12 Brian J. Tarricone (not reading bugmail) 2008-10-03 17:19:10 CEST
(In reply to comment #11)
> (In reply to comment #10)
> > PLEASE test.
> 
> Does not work out here (Xfce 4.4.2 on Fedora 9):
> With the second patch (attachment #1817 ) I noticed some applcaions starting
> twice: Once from /etc/xdg/autostart and once from xfce4-session, although I had
> _not_ selected to save my session and wiped the session cache.

I don't see how that could have anything to do with my patch.  I didn't touch any of the code dealing with saving the session, just loading it.  On next startup (when you end up with multiple copies of things), what's in your session file?

> With the parallel startup patch v1 (attachment #1849 ) it's even worse: The
> startup order seems not predictable, so the panels have a different layout
> every time I log in. I already had plugins (places, xfce-menu) sitting _in_ the
> tray instead of the panel!

Right, as I said, the v1 patch is bad because startup order doesn't follow priority order.

But your panel issues have nothing to do with the patch -- startup order is irrelevant where panel layout is concerned.  Plugins can't go in the systray, period.  You're either mistaken, or you've hit a weird-ass panel bug that's being triggered by something unrelated.
Comment 13 Brian J. Tarricone (not reading bugmail) 2008-10-03 17:20:09 CEST
(In reply to comment #11)

> Does not work out here (Xfce 4.4.2 on Fedora 9):

Oh, and are you using xfce4-session from SVN, or are you patching 4.4.2?  4.4.2 *should* be ok, but... I'd prefer if people can test against SVN.
Comment 14 Brian J. Tarricone (not reading bugmail) 2008-10-07 08:51:08 CEST
I'm actually pretty happy with how this is working, and others seem to think it works well, too, so let's consider this fixed.  If there are issues, please open a new bug.

Bug #4312

Reported by:
Auke Kok
Reported on: 2008-08-22
Last modified on: 2009-07-14

People

Assignee:
Brian J. Tarricone (not reading bugmail)
CC List:
8 users

Version

Attachments

make subprocess startup async. (1.29 KB, patch)
2008-08-22 20:45 CEST , Auke Kok
no flags
first cut at real parallel startup (16.14 KB, patch)
2008-09-15 08:38 CEST , Brian J. Tarricone (not reading bugmail)
no flags
parallel startup while respecting priority order (17.11 KB, patch)
2008-09-30 09:34 CEST , Brian J. Tarricone (not reading bugmail)
no flags

Additional information