[Twisted-Python] Slow data transfer with Twisted + socat + Windows

[Jason Heeris]

I'm writing an application with Twisted/GTK to manage a serial device,
for use on both Windows and Linux. Unfortunately I can't use serial
comms directly under Windows[1], so my workaround is to launch
socat[2] in a subprocess to relay data between a local TCP port and
the serial port. The socat command I use is:

socat TCP4-LISTEN:31415 /dev/com4,raw,echo=0,b57600

I can then send and receive data over localhost:31415 same as I would
for a serial port.

The trouble is, a transaction that takes a minute under Linux (using
*either* direct serial comms or the same socat subprocess approach)
now takes more than 50 times as long under Windows.

I've attached the script I'm using to test, mainly so it's clear what
I'm doing — although it's useless unless you happen to have a serial
device using a protocol you can shoehorn into that script. I've also
attached the output where you can see the difference between different
reactors on different platforms. (In my app I use
reactor.spawnProcess(...), but I'm not doing that here.)

Basically, under Linux either approach takes 4 seconds; under Windows
the SelectReactor takes 30s, and the gtk2reactor takes 220s!

Profiling the "reactor.run()" call on the different platforms (both
for the select reactor) showed that (a) most time was spent in the
select.select call, and (b) the time spent in select was 8 times more
under Windows than Linux. I have no idea if this is useful information
or not.

I'm pretty sure it's not a socat-specific issue, because I get the
same problem if I use com2tcp[3]. It could be Cygwin related, but I
don't know how determine that for sure. Also, the fact that it runs
slower using the gtk2reactor (vs. selectreactor) suggests to me that
it could be my code, not theirs.

I found an old Cygwin thread[4] that claimed setting NODELAY on the
listening socket helped with a similar problem, but using

socat TCP4-LISTEN:31415,nodelay [etc]

...made no difference.

So I'm a bit stuck. Does anyone know how I can narrow it down further?
Has someone else noticed slow TCP behaviour under Windows? Or Cygwin
utilities? Have I made some glaringly obvious mistake in my script
there?

Any help appreciated. :)

Cheers,
Jason

[1] http://twistedmatrix.com/trac/ticket/4862
[2] http://www.dest-unreach.org/socat/
[3] http://com0com.sourceforge.net/
[4] http://www.mail-archive.com/***@cygwin.com/msg66791.html

[Michael Thompson]

Post by Jason Heeris
I'm writing an application with Twisted/GTK to manage a serial device,
for use on both Windows and Linux. Unfortunately I can't use serial
comms directly under Windows[1], so my workaround is to launch
socat[2] in a subprocess to relay data between a local TCP port and
   socat TCP4-LISTEN:31415 /dev/com4,raw,echo=0,b57600
I can then send and receive data over localhost:31415 same as I would
for a serial port.
The trouble is, a transaction that takes a minute under Linux (using
*either* direct serial comms or the same socat subprocess approach)
now takes more than 50 times as long under Windows.
I've attached the script I'm using to test, mainly so it's clear what
I'm doing — although it's useless unless you happen to have a serial
device using a protocol you can shoehorn into that script. I've also
attached the output where you can see the difference between different
reactors on different platforms. (In my app I use
reactor.spawnProcess(...), but I'm not doing that here.)
Basically, under Linux either approach takes 4 seconds; under Windows
the SelectReactor takes 30s, and the gtk2reactor takes 220s!
Profiling the "reactor.run()" call on the different platforms (both
for the select reactor) showed that (a) most time was spent in the
select.select call, and (b) the time spent in select was 8 times more
under Windows than Linux. I have no idea if this is useful information
or not.
I'm pretty sure it's not a socat-specific issue, because I get the
same problem if I use com2tcp[3]. It could be Cygwin related, but I
don't know how determine that for sure. Also, the fact that it runs
slower using the gtk2reactor (vs. selectreactor) suggests to me that
it could be my code, not theirs.
I found an old Cygwin thread[4] that claimed setting NODELAY on the
listening socket helped with a similar problem, but using
   > socat TCP4-LISTEN:31415,nodelay [etc]
...made no difference.
So I'm a bit stuck. Does anyone know how I can narrow it down further?
Has someone else noticed slow TCP behaviour under Windows? Or Cygwin
utilities? Have I made some glaringly obvious mistake in my script
there?

Does you program just write via TCP to this serial device, it doesn't
do anything else that could block?

Have you tried running this under different reactors?

Running the windows program against the Linux socat device might help
to reduce the number of variables.

Michael

[Jason Heeris]

Post by Michael Thompson
Does you program just write via TCP to this serial device, it doesn't
do anything else that could block?

Both the "bare bones" script and the real app writes and reads from
the device, so I wouldn't expect full speed communications. But I
can't see anything else in my script that might actually block.

Post by Michael Thompson
Have you tried running this under different reactors?

Yep — no difference under Linux, but the GTK reactor is slower than
the select reactor under Windows (see original email for times).

Post by Michael Thompson
Running the windows program against the Linux socat device might help
to reduce the number of variables.

Aha! Didn't think of that, but interesting results...

socat on Linux, socat_test.py on Windows VM:
- select reactor: 4s
- win32reactor: 4s
- gtk reactor: 220s

socat on Windows VM, socat_test.py on Linux:
- select reactor: 30s
- gtk reactor: 30s

Interesting, but not exactly clarifying...

— Jason

[Michael Thompson]

---------- Forwarded message ----------
From: Michael Thompson <***@gmail.com>
Date: 9 March 2011 07:02
Subject: Re: [Twisted-Python] Slow data transfer with Twisted + socat + Windows

Post by Jason Heeris

Post by Michael Thompson
Does you program just write via TCP to this serial device, it doesn't
do anything else that could block?

Both the "bare bones" script and the real app writes and reads from
the device, so I wouldn't expect full speed communications. But I
can't see anything else in my script that might actually block.

Post by Michael Thompson
Have you tried running this under different reactors?

Yep — no difference under Linux, but the GTK reactor is slower than
the select reactor under Windows (see original email for times).

Post by Michael Thompson
Running the windows program against the Linux socat device might help
to reduce the number of variables.

Aha! Didn't think of that, but interesting results...
 - select reactor: 4s
 - win32reactor: 4s
 - gtk reactor: 220s
 - select reactor: 30s
 - gtk reactor: 30s
Interesting, but not exactly clarifying...

And from your first email

socat on Windows, socat_test.py on Windows VM

select 30s
gtk reactor 220s

My read of these results is that socat on windows limits the
performance to 30s and in addition the gtkreactor on windows has
performance problems.

The gtkreactor on windows has a different implementation which seems
to be missing any way for the GTK loop to notify the reactor of IO.
You could try changing the timeout from 0.1 to something smaller, but
that will probably hammer the CPU when idle.

pyserial includes an example that makes a serial port accessible of
TCP you could see if that is any faster than socat.

Perhaps you can write a serial to TCP adaptor that encapsulates the
serial logic so reduce the IO between this application and the GTK app
to a message like start_sync, stop_sync.

Michael

[Jason Heeris]

Post by Michael Thompson
My read of these results is that socat on windows limits the
performance to 30s and in addition the gtkreactor on windows has
performance problems.

I'd say it's Cygwin, not socat, but yes.

Post by Michael Thompson
pyserial includes an example that makes a serial port accessible of
TCP you could see if that is any faster than socat.

Okay, I tried that, and with the select reactor, it's great! 3s flat!

With the GTK reactor... still 220s. So, no luck.

Post by Michael Thompson
Perhaps you can write a serial to TCP adaptor that encapsulates the
serial logic so reduce the IO between this application and the GTK app
to a message like start_sync, stop_sync.

I've only shown the simplest possible protocol here - if I really did
encapsulate all the possible commands and interactions in a
synchronous subprocess, I'd basically be writing all the protocols
twice.

Besides, none of that will help if the GTK reactor can't communicate
with anything properly.

Cheers,
Jason

[Jason Heeris]

Post by Jason Heeris
Besides, none of that will help if the GTK reactor can't communicate
with anything properly.

I adapted my original script to simply work with an echo server, and
there really does seem to be a problem with the GTK reactor, no matter
what the transport.

Is this worth filing a bug about (since I have a neat little demo
script to illustrate the problem), or is it just too vague?

- Jason

[Itamar Turner-Trauring]

Post by Jason Heeris
Is this worth filing a bug about (since I have a neat little demo
script to illustrate the problem), or is it just too vague?

It's worth filing a bug, yes.