Speed up your Fabric deployments with Parex (opens in new tab)

(paulbohm.com)

19 pointsenki15y ago8 comments

8 comments

6 comments · 2 top-level

njharman15y ago· 3 in thread

Why isn't Fabric doing this? This would be basic feature of deployment software I would think. I vaguely remember the old, old Fabric (before they rewrote everything) did this.

jacobian15y ago

There's a branch with this feature: http://github.com/goosemo/fabric. The current work's in the "mutliprocessing" branch. I'm using it for a big deploy right now, and it appears to be working well.

[edit: fixed my mistake about where the most current code lives]

enkiOP15y ago

i think they've been planning to for a while: http://code.fabfile.org/issues/show/19

i haven't really followed their progress too closely, but they seem to be close to releasing something for 1.0.

I think they do client-side threading while i prefer to to do fork() on the server to avoid multiple ssh connections and retain maximum control.

i'll reevaluate when they ship. till then parex works for me.

goosemo15y ago

The branch actually does client side forks, one for each server in a task. Then each call inside a task is run sequentially, as expected in normal fabric usage. I've also included the ability to set a pool size, so that one can manage the number of ssh connections open simultaneously.

1 more reply

goosemo15y ago· 1 in thread

Just to note/ask though, this requires that you have a python script on the server that uses this as a lib? and requires tornado?

enkiOP15y ago

i just use the same fabfile on the server. and doesn't require tornado on linux (if epoll is available)

j / k navigate · click thread line to collapse

8 comments

6 comments · 2 top-level

njharman15y ago· 3 in thread

Why isn't Fabric doing this? This would be basic feature of deployment software I would think. I vaguely remember the old, old Fabric (before they rewrote everything) did this.

jacobian15y ago

There's a branch with this feature: http://github.com/goosemo/fabric. The current work's in the "mutliprocessing" branch. I'm using it for a big deploy right now, and it appears to be working well.

[edit: fixed my mistake about where the most current code lives]

enkiOP15y ago

i think they've been planning to for a while: http://code.fabfile.org/issues/show/19

i haven't really followed their progress too closely, but they seem to be close to releasing something for 1.0.

I think they do client-side threading while i prefer to to do fork() on the server to avoid multiple ssh connections and retain maximum control.

i'll reevaluate when they ship. till then parex works for me.

goosemo15y ago

The branch actually does client side forks, one for each server in a task. Then each call inside a task is run sequentially, as expected in normal fabric usage. I've also included the ability to set a pool size, so that one can manage the number of ssh connections open simultaneously.

1 more reply

goosemo15y ago· 1 in thread

Just to note/ask though, this requires that you have a python script on the server that uses this as a lib? and requires tornado?

enkiOP15y ago

i just use the same fabfile on the server. and doesn't require tornado on linux (if epoll is available)

j / k navigate · click thread line to collapse