Discussion:
[Twisted-Python] pywin32 -> cffi update and feedback request
Oliver Palmer
2016-01-06 23:35:51 UTC
Permalink
So expanding on the "Should the CFFI wrapper and functions for win32 be a
separate project" thread from six months ago I think I'd like some
feedback. The pywincffi <https://github.com/opalmer/pywincffi> project,
which I'm planning to use to start replacing pywin32 in Twisted, is getting
close to its first release. At this point there's enough code in place
that it could probably be used to replace pywin32 in
twisted.python.lockfile and other parts of pywin32 in Twisted. Before
proposing any patches however, I'd like to take this opportunity to welcome
feedback from people on this list. Although pywincffi will not be a
'Twisted project' Twisted will be the primary consumer of pywincffi so I'd
like to make sure developers here are happy with the direction that has
been taken.

For some background, the core objectives and intended design features are
below (nothing all that special mostly):

* It should be easy to build and retrieve the binary files (wheels for now,
easy to add more later).
* Python 2.6, 2.7 and 3.x are supported from a single code base.
* Type conversion, error checking and other 'C like' code should be the
responsibility of the library where possible.
* APIs provided by pywincffi should mirror their Windows counterparts as
closely as possible so the MSDN documentation can be more easily used as
reference.
* For contributors, it should be possible to work on any platform. It
should also be possible to contribute without having to manually build a VM.
* For consumers, documentation and error messages should be descriptive,
consistent, complete and accessible. Examples should be provided for more
complex use cases.

From a functionality and design standpoint, I think the above are more or
less achieved and can be maintained going forward. With that in mind, I'd
like to know if anyone here has other ideas that they believe should be
incorporated. Of course if anyone happens to look at the code and find
functional issues with it now would be a good time to address those issues
too.

Thanks in advance for the help!

---Oliver
Glyph Lefkowitz
2016-01-06 23:44:43 UTC
Permalink
So expanding on the "Should the CFFI wrapper and functions for win32 be a separate project" thread from six months ago I think I'd like some feedback. The pywincffi <https://github.com/opalmer/pywincffi> project, which I'm planning to use to start replacing pywin32 in Twisted, is getting close to its first release. At this point there's enough code in place that it could probably be used to replace pywin32 in twisted.python.lockfile and other parts of pywin32 in Twisted. Before proposing any patches however, I'd like to take this opportunity to welcome feedback from people on this list. Although pywincffi will not be a 'Twisted project' Twisted will be the primary consumer of pywincffi so I'd like to make sure developers here are happy with the direction that has been taken.
* It should be easy to build and retrieve the binary files (wheels for now, easy to add more later).
* Python 2.6, 2.7 and 3.x are supported from a single code base.
* Type conversion, error checking and other 'C like' code should be the responsibility of the library where possible.
* APIs provided by pywincffi should mirror their Windows counterparts as closely as possible so the MSDN documentation can be more easily used as reference.
* For contributors, it should be possible to work on any platform. It should also be possible to contribute without having to manually build a VM.
* For consumers, documentation and error messages should be descriptive, consistent, complete and accessible. Examples should be provided for more complex use cases.
From a functionality and design standpoint, I think the above are more or less achieved and can be maintained going forward. With that in mind, I'd like to know if anyone here has other ideas that they believe should be incorporated. Of course if anyone happens to look at the code and find functional issues with it now would be a good time to address those issues too.
Thanks in advance for the help!
This sounds absolutely fantastic - thank you for taking this on.

Something else that would be nice to keep in mind - I would really like to eliminate our Pyrex support files for IOCP as well; if you could make sure those APIs are wrapped by pywincffi (I think they might be missing from pywin32, or at least, I believe they were at one point, which is why Pavel wrote his own thing), and perhaps even contribute patches to eliminate that old and crufty code, that would be fantastic :).

-glyph
Amber Brown
2016-01-07 02:49:59 UTC
Permalink
Worth noting, I have a CFFI binding for IOCP already (its blocked on base support for Windows on Py3, that ticket is up for review), so it's not needed right now. Although, ideally, replacing our custom built thing with a library would be the best case, but we can put more important things first :)
Post by Glyph Lefkowitz
So expanding on the "Should the CFFI wrapper and functions for win32 be a separate project" thread from six months ago I think I'd like some feedback.  The pywincffi project, which I'm planning to use to start replacing pywin32 in Twisted, is getting close to its first release.  At this point there's enough code in place that it could probably be used to replace pywin32 in twisted.python.lockfile and other parts of pywin32 in Twisted.  Before proposing any patches however, I'd like to take this opportunity to welcome feedback from people on this list.  Although pywincffi will not be a 'Twisted project' Twisted will be the primary consumer of pywincffi so I'd like to make sure developers here are happy with the direction that has been taken.
* It should be easy to build and retrieve the binary files (wheels for now, easy to add more later).
* Python 2.6, 2.7 and 3.x are supported from a single code base.
* Type conversion, error checking and other 'C like' code should be the responsibility of the library where possible.
* APIs provided by pywincffi should mirror their Windows counterparts as closely as possible so the MSDN documentation can be more easily used as reference.
* For contributors, it should be possible to work on any platform.  It should also be possible to contribute without having to manually build a VM.
* For consumers, documentation and error messages should be descriptive, consistent, complete and accessible.  Examples should be provided for more complex use cases.
From a functionality and design standpoint, I think the above are more or less achieved and can be maintained going forward.  With that in mind, I'd like to know if anyone here has other ideas that they believe should be incorporated.  Of course if anyone happens to look at the code and find functional issues with it now would be a good time to address those issues too.
Thanks in advance for the help!
This sounds absolutely fantastic - thank you for taking this on.
Something else that would be nice to keep in mind - I would really like to eliminate our Pyrex support files for IOCP as well; if you could make sure those APIs are wrapped by pywincffi (I think they might be missing from pywin32, or at least, I believe they were at one point, which is why Pavel wrote his own thing), and perhaps even contribute patches to eliminate that old and crufty code, that would be fantastic :).
-glyph
Zooko Wilcox-O'Hearn
2016-01-13 01:44:39 UTC
Permalink
FYI, this would, if it were compatible with Tahoe's packaging, fix
these outstanding issues for Tahoe:

* https://tahoe-lafs.org/trac/tahoe-lafs/ticket/142# pywin32 can't be
installed automatically

* https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2028# Twisted
endpoints introduce a dependency on pywin32

Sincerely,

Zooko
Oliver Palmer
2016-01-14 01:59:24 UTC
Permalink
Sounds like pywincffi should fix that issue....I'm planning on distributing
binary wheels which pip and setuptools should be able to install like any
other dependency. I can make sure to build other formats (exe, msi, egg,
etc) if it helps though the binary wheels should be enough to avoid the
same installation issues pywin32 has in most cases. Is Tahoe's packaging
going to require anything special that I should account for or will Twisted
depending on pywincffi work for Tahoe?
Post by Zooko Wilcox-O'Hearn
FYI, this would, if it were compatible with Tahoe's packaging, fix
* https://tahoe-lafs.org/trac/tahoe-lafs/ticket/142# pywin32 can't be
installed automatically
* https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2028# Twisted
endpoints introduce a dependency on pywin32
Sincerely,
Zooko
_______________________________________________
Twisted-Python mailing list
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Oliver Palmer
2016-01-24 01:18:40 UTC
Permalink
So 0.1.2 is out and I've uploaded a patch set for review to #7889 which
replaces win32api.FormatMessage and removes the dependency on pywin32 in
twisted.python.lockfile (though it looks like we can do without needing to
call on pywincffi in this case).

Unfortunately, my update was spam filtered with a 57.36% probability of
being spam. I'm human, I promise.
Post by Oliver Palmer
Sounds like pywincffi should fix that issue....I'm planning on
distributing binary wheels which pip and setuptools should be able to
install like any other dependency. I can make sure to build other formats
(exe, msi, egg, etc) if it helps though the binary wheels should be enough
to avoid the same installation issues pywin32 has in most cases. Is
Tahoe's packaging going to require anything special that I should account
for or will Twisted depending on pywincffi work for Tahoe?
Post by Zooko Wilcox-O'Hearn
FYI, this would, if it were compatible with Tahoe's packaging, fix
* https://tahoe-lafs.org/trac/tahoe-lafs/ticket/142# pywin32 can't be
installed automatically
* https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2028# Twisted
endpoints introduce a dependency on pywin32
Sincerely,
Zooko
_______________________________________________
Twisted-Python mailing list
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Glyph Lefkowitz
2016-01-24 05:01:42 UTC
Permalink
So 0.1.2 is out and I've uploaded a patch set for review to #7889 which replaces win32api.FormatMessage and removes the dependency on pywin32 in twisted.python.lockfile (though it looks like we can do without needing to call on pywincffi in this case).
Unfortunately, my update was spam filtered with a 57.36% probability of being spam. I'm human, I promise.
I've trained the spam database to correct the error, hopefully you can submit it again.

-glyph
Oliver Palmer
2016-01-24 06:09:26 UTC
Permalink
Worked that time, thanks Glyph.
Post by Oliver Palmer
Post by Oliver Palmer
So 0.1.2 is out and I've uploaded a patch set for review to #7889 which
replaces win32api.FormatMessage and removes the dependency on pywin32 in
twisted.python.lockfile (though it looks like we can do without needing to
call on pywincffi in this case).
Post by Oliver Palmer
Unfortunately, my update was spam filtered with a 57.36% probability of
being spam. I'm human, I promise.
I've trained the spam database to correct the error, hopefully you can submit it again.
-glyph
_______________________________________________
Twisted-Python mailing list
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Loading...