[Twisted-Python] Windows buildbots and Appveyor

[Craig Rodrigues]

The Appveyor setup with Python 2.7 currently does not build any C
extensions.
This means that IOCP reactor is not currently built and none of the IOCP
tests are run in Appveyor. I'm not sure offhand what the fix is,
but I notice that quite a few Python projects import this script from
Olivier Grisel:
https://github.com/ogrisel/python-appveyor-demo/blob/master/appveyor/run_with_env.cmd

which sets the environment correctly for the Windows compilers, and
then uses the script from inside appveyor.yml with something like:
https://github.com/ogrisel/python-appveyor-demo/blob/master/appveyor.yml

It would be nice if we could get the C extensions buildin in Appveyor.


The Appveyor setup does not have a Python 3 setup, so no Python 3 builds
are done.

There used to be a Python 3 buildbot, and although many of the tests were
broken,
it was producing some interesting logs. For example, I saw a lot of
warnings
in the SSL code where strings where being passed as arguments to API's which
took bytes as arguments.

Can we bring back at least one Windows Python 3 buildbot, and leave
it as an unsupported, on-demand environment?

I am working on getting more Windows Python 3 tests to pass, so this
would be useful.


--
Craig

[Oliver Palmer]

If you or someone else ends up working on this I can attest that his setup
works well and will build C-extensions. I've been using his script
extensively on pywincffi for Python 2.6+ and Python 3.3+ so far without
issues. For reference, here's my yaml file:

https://github.com/opalmer/pywincffi/blob/master/appveyor.yml

Since Twisted is using tox I assume that either tox should setup the
environment or %PYTHON_VERSION% and %PYTHON_ARCH% need to be set by the
build matrix and tox would be called with the wrapper:

cmd /E:ON /V:ON /C run_with_cmd.cmd tox <arguments>

For pywincffi I thought about writing a tox plugin to do the work of
setting up the environment but ended up deciding against it and using the
build matrix approach instead. It kept it simple, made it easy for people
to see how the build was setup and adding new versions to the build became
a one or two line change.

I'm not sure about how others would feel about it but it seems like
expanding the build matrix and adding run_with_cmd would be a good way to
start testing against more versions and fixing the C-extension issue.
Improving the build on AppyVeyor might also help shed some light on the
existing test failures too maybe.

Post by Craig Rodrigues
The Appveyor setup with Python 2.7 currently does not build any C
extensions.
This means that IOCP reactor is not currently built and none of the IOCP
tests are run in Appveyor. I'm not sure offhand what the fix is,
but I notice that quite a few Python projects import this script from
https://github.com/ogrisel/python-appveyor-demo/blob/master/appveyor/run_with_env.cmd
which sets the environment correctly for the Windows compilers, and
https://github.com/ogrisel/python-appveyor-demo/blob/master/appveyor.yml
It would be nice if we could get the C extensions buildin in Appveyor.
The Appveyor setup does not have a Python 3 setup, so no Python 3 builds
are done.
There used to be a Python 3 buildbot, and although many of the tests were
broken,
it was producing some interesting logs. For example, I saw a lot of
warnings
in the SSL code where strings where being passed as arguments to API's which
took bytes as arguments.
Can we bring back at least one Windows Python 3 buildbot, and leave
it as an unsupported, on-demand environment?
I am working on getting more Windows Python 3 tests to pass, so this
would be useful.
--
Craig
_______________________________________________
Twisted-Python mailing list
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Craig Rodrigues]

Post by Oliver Palmer
If you or someone else ends up working on this I can attest that his setup
works well and will build C-extensions. I've been using his script
extensively on pywincffi for Python 2.6+ and Python 3.3+ so far without
https://github.com/opalmer/pywincffi/blob/master/appveyor.yml
Since Twisted is using tox I assume that either tox should setup the
environment or %PYTHON_VERSION% and %PYTHON_ARCH% need to be set by the
cmd /E:ON /V:ON /C run_with_cmd.cmd tox <arguments>

Using tox is the desired way to go.
I made an initial cut at this here:
https://github.com/rodrigc/twisted/blob/appveyor2/appveyor.yml

Do you have any cycles to help me improve this and get something working?

--
Craig

[Oliver Palmer]

I should this weekend but it looks like you're on the right track so far.
Looking at tox's config it shouldn't have issues picking up the environment
being added in appveyor.yaml because passenv = * currently. If this works
out then we can hopefully avoid the need for a plugin and just rely
directly on the wrapper script. At least in my experience the wrapper
script was all I needed to get builds working across Python versions.

By the way, this can be really useful to help debug problems at runtime
while the build is running:

https://www.appveyor.com/docs/how-to/rdp-to-build-worker

Hit me with @opalmer on GitHub if you've got a PR open between now and the
weekend or want some input, I'll be happy to help either there or on this
list.

Post by Craig Rodrigues

Post by Oliver Palmer
If you or someone else ends up working on this I can attest that his
setup works well and will build C-extensions. I've been using his script
extensively on pywincffi for Python 2.6+ and Python 3.3+ so far without
https://github.com/opalmer/pywincffi/blob/master/appveyor.yml
Since Twisted is using tox I assume that either tox should setup the
environment or %PYTHON_VERSION% and %PYTHON_ARCH% need to be set by the
cmd /E:ON /V:ON /C run_with_cmd.cmd tox <arguments>

Using tox is the desired way to go.
https://github.com/rodrigc/twisted/blob/appveyor2/appveyor.yml
Do you have any cycles to help me improve this and get something working?
--
Craig
_______________________________________________
Twisted-Python mailing list
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python