Discussion:
[Twisted-Python] Twisted 16.7.0rc1 Release Candidate Announcement
ex vito
2016-12-23 11:49:35 UTC
Permalink
[...]
python -m pip install Twisted==16.7.0rc1
Please test it, and let me know how your applications fare, good or bad! If nothing comes up, 16.7 looks like it'll land just before the new year.
Amber,

Do you think https://github.com/twisted/twisted/pull/599 can go in?
As far as I can tell, it's been reviewed and approved for merging. If not, please point out what should be improved.

Thanks and Happy Holidays,
--
exvito
Glyph Lefkowitz
2016-12-23 20:31:10 UTC
Permalink
Post by ex vito
[...]
python -m pip install Twisted==16.7.0rc1
Please test it, and let me know how your applications fare, good or bad! If nothing comes up, 16.7 looks like it'll land just before the new year.
Amber,
Do you think https://github.com/twisted/twisted/pull/599 <https://github.com/twisted/twisted/pull/599> can go in?
As far as I can tell, it's been reviewed and approved for merging. If not, please point out what should be improved.
Amber is the release manager and therefore has final say, but, generally, we try not to roll more things into releases. Instead, we try to roll releases more frequently (which has been a resounding success during 2016, and hopefully we can keep that momentum during 2017).

However, I have to say I'm very sorry this did not get merged in a timely way! It seems I didn't get the github notification when you @mentioned me; I have no idea why that would be.

I'll get test results on trunk and get it merged, and it should be in the next 17.x release.

-glyph
Glyph Lefkowitz
2016-12-23 21:22:09 UTC
Permalink
Post by Glyph Lefkowitz
Post by ex vito
[...]
python -m pip install Twisted==16.7.0rc1
Please test it, and let me know how your applications fare, good or bad! If nothing comes up, 16.7 looks like it'll land just before the new year.
Amber,
Do you think https://github.com/twisted/twisted/pull/599 <https://github.com/twisted/twisted/pull/599> can go in?
As far as I can tell, it's been reviewed and approved for merging. If not, please point out what should be improved.
Amber is the release manager and therefore has final say, but, generally, we try not to roll more things into releases. Instead, we try to roll releases more frequently (which has been a resounding success during 2016, and hopefully we can keep that momentum during 2017).
I'll get test results on trunk and get it merged, and it should be in the next 17.x release.
-glyph
It's merged now: https://twistedmatrix.com/trac/ticket/8911#comment:5 <https://twistedmatrix.com/trac/ticket/8911#comment:5>

On a related note - this still linked the '#' syntax back to the ticket number in Trac.

Perhaps related - tomprince fixed the bugs that were preventing us from upgrading us to 1.2, and upgraded us to 1.2. So perhaps this has introduced a problem :).

-glyph
ex vito
2016-12-24 14:01:44 UTC
Permalink
Post by Glyph Lefkowitz
Post by Glyph Lefkowitz
Post by ex vito
[...]
python -m pip install Twisted==16.7.0rc1
Please test it, and let me know how your applications fare, good or bad! If nothing comes up, 16.7 looks like it'll land just before the new year.
Amber,
Do you think https://github.com/twisted/twisted/pull/599 can go in?
As far as I can tell, it's been reviewed and approved for merging. If not, please point out what should be improved.
Amber is the release manager and therefore has final say, but, generally, we try not to roll more things into releases. Instead, we try to roll releases more frequently (which has been a resounding success during 2016, and hopefully we can keep that momentum during 2017).
I'll get test results on trunk and get it merged, and it should be in the next 17.x release.
Glyph,

The next 17.x release sounds good. I'm in no hurries and that keeps an open time window to address the very tightly related https://twistedmatrix.com/trac/ticket/8912 which I recently noted was created by Tom Prince.
Post by Glyph Lefkowitz
It's merged now: https://twistedmatrix.com/trac/ticket/8911#comment:5
On a related note - this still linked the '#' syntax back to the ticket number in Trac.
Perhaps related - tomprince fixed the bugs that were preventing us from upgrading us to 1.2, and upgraded us to 1.2. So perhaps this has introduced a problem :).
Thanks for the quick feedback and heads up on the '#' link.
Regards,
--
exvito
Glyph Lefkowitz
2016-12-24 21:29:16 UTC
Permalink
Post by ex vito
Post by Glyph Lefkowitz
Post by Glyph Lefkowitz
Post by ex vito
[...]
python -m pip install Twisted==16.7.0rc1
Please test it, and let me know how your applications fare, good or bad! If nothing comes up, 16.7 looks like it'll land just before the new year.
Amber,
Do you think https://github.com/twisted/twisted/pull/599 <https://github.com/twisted/twisted/pull/599> can go in?
As far as I can tell, it's been reviewed and approved for merging. If not, please point out what should be improved.
Amber is the release manager and therefore has final say, but, generally, we try not to roll more things into releases. Instead, we try to roll releases more frequently (which has been a resounding success during 2016, and hopefully we can keep that momentum during 2017).
I'll get test results on trunk and get it merged, and it should be in the next 17.x release.
Glyph,
The next 17.x release sounds good. I'm in no hurries and that keeps an open time window to address the very tightly related https://twistedmatrix.com/trac/ticket/8912 <https://twistedmatrix.com/trac/ticket/8912> which I recently noted was created by Tom Prince.
Great. Basically, try not to worry about releases too much - just do the work as quick as you can, do reviews to speed along the queue in general, and the ever-faster release cadence that Amber has put in place will catch your changes soon enough.
Post by ex vito
Post by Glyph Lefkowitz
It's merged now: https://twistedmatrix.com/trac/ticket/8911#comment:5 <https://twistedmatrix.com/trac/ticket/8911#comment:5>
On a related note - this still linked the '#' syntax back to the ticket number in Trac.
Perhaps related - tomprince fixed the bugs that were preventing us from upgrading us to 1.2, and upgraded us to 1.2. So perhaps this has introduced a problem :).
Thanks for the quick feedback and heads up on the '#' link.
This was a heads-up for Jean-Paul, who has been trying to change this behavior (to make '#' on new commits refer to Github issues / PRs, and 'ticket:' refer to Trac tickets, so we don't get spurious email notifications about old Trac tickets). You can pretty much ignore it :).

-glyph
Jean-Paul Calderone
2016-12-27 00:14:04 UTC
Permalink
Post by Glyph Lefkowitz
It's merged now: https://twistedmatrix.com/trac/ticket/8911#comment:5
On a related note - this still linked the '#' syntax back to the ticket number in Trac.
Perhaps related - tomprince fixed the bugs that were preventing us from
upgrading us to 1.2, and upgraded us to 1.2. So perhaps this has
introduced a problem :).
-glyph
It looks like the link to ticket 599 in the merge/resolve comment on ticket
8911 is a different kind of link than the one I was looking in to. I agree
it would be nice to fix this, too, and perhaps I'll take a look. My goal
was only to stop ticket 599 (for example) from also receiving a copy of
this comment - which appears to have succeeded. Let me know if I've missed
something.

Jean-Paul
Glyph Lefkowitz
2016-12-29 13:28:44 UTC
Permalink
Post by Glyph Lefkowitz
It's merged now: https://twistedmatrix.com/trac/ticket/8911#comment:5 <https://twistedmatrix.com/trac/ticket/8911#comment:5>
On a related note - this still linked the '#' syntax back to the ticket number in Trac.
Perhaps related - tomprince fixed the bugs that were preventing us from upgrading us to 1.2, and upgraded us to 1.2. So perhaps this has introduced a problem :).
-glyph
It looks like the link to ticket 599 in the merge/resolve comment on ticket 8911 is a different kind of link than the one I was looking in to. I agree it would be nice to fix this, too, and perhaps I'll take a look. My goal was only to stop ticket 599 (for example) from also receiving a copy of this comment - which appears to have succeeded. Let me know if I've missed something.
Sorry - this is fine. I mistakenly assumed that the fact that the link was highlighted meant that the comment happened. If we were to kill '#' links, that would also break all the existing commit messages, which would be bad. So please leave it as-is!

-glyph
John Santos
2016-12-28 23:20:27 UTC
Permalink
Seems to require a new Python module, Automat. (Not mentioned in the
NEWS file, INSTALL, etc.) I found Automat 0.3.0 (with Glyph's
fingerprints on it) at PyPI, but it only exists in .whl format.
Downloaded it, but I'm not sure what to do next. I was able to get it
working by downloading and installing get-pip.py and Automat (contrary
to the python.org documentation, I didn't have pip installed on my
system even though it is Python 2.7.13 downloaded from python.org.)
sudo python get-pip.py
sudo pip install Automat

This is worrisome to me because I think this requires Internet access to
pypi.python.org, which I have on my development and testing systems, but
not on my production systems. Everything else I've ever needed, I could
download and install source files.

Having done this, my app seems to run fine (only quick sanity testing so
far.)


----install---
Searching for Automat>=0.3.0
Reading https://pypi.python.org/simple/Automat/
Download error on https://pypi.python.org/simple/Automat/: [SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590) --
Some packages may not be found!
Couldn't find index page for 'Automat' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
Download error on https://pypi.python.org/simple/: [SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590) --
Some packages may not be found!
No local packages or working download links found for Automat>=0.3.0
error: Could not find suitable distribution for
Requirement.parse('Automat>=0.3.0')
------

---try to use without Automat---
***@Medusa:~/run> python medusa_server.py Rain
Traceback (most recent call last):
File "medusa_server.py", line 72, in <module>
from twisted.application import service, internet
File
"/usr/lib/python2.7/site-packages/Twisted-16.7.0rc1-py2.7-linux-x86_64.egg/twisted/application/internet.py",
line 54, in <module>
from automat import MethodicalMachine
ImportError: No module named automat
------
Hello everyone, and happy holidays!
Although it didn't quite fall on the solstice, one could call this
Twisted release a little ray of sunshine, with some nice new
- twisted.web.client.Agent now supports IPv6! It's also now the
primary web client in Twisted, with twisted.web.client.getPage being
deprecated in favour of it and Treq.
- twisted.internet.ssl.CertificateOptions has had its `method`
argument deprecated, in favour of the new raiseMinimumTo,
lowerMaximumSecurityTo, and insecurelyLowerMinimumTo arguments, which
take TLSVersion arguments. This allows you to better give a range of
versions of TLS you wish to negotiate, rather than forcing yourself to
any one version.
- twisted.internet.ssl.CertificateOptions will use OpenSSL's
MODE_RELEASE_BUFFERS, which will let it free unused memory that was
held by idle TLS connections.
- You can now call the new twist runner with `python -m twisted`.
- Conch has the beginnings of ECDH key exchange, and supports
`hmac-sha2-384` in twisted.conch.ssh.
- Better Unicode support in twisted.internet.reactor.spawnProcess,
especially on Windows on Python 3.6.
- More Python 3 porting in Conch, and more under-the-hood changes to
facilitate a Twisted-wide jump to new-style classes only on Python 2
in 2018/2019. This release has also been tested on Python 3.6 on Linux.
- Lots of deprecated code removals, to make a sleeker, less confusing
Twisted.
- 20+ closed tickets.
You can get the tarball and the NEWS file at
https://twistedmatrix.com/Releases/rc/16.7.0rc1/ , or you can try it
python -m pip install Twisted==16.7.0rc1
Please test it, and let me know how your applications fare, good or
bad! If nothing comes up, 16.7 looks like it'll land just before the
new year.
With festive hoots,
Amber Brown
_______________________________________________
Twisted-Python mailing list
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
--
John Santos
Evans Griffiths & Hart, Inc.
781-861-0670 ext 539
Amber "Hawkie" Brown
2016-12-28 23:37:28 UTC
Permalink
Hi John,

.whl is just an optimised binary format, and can be installed by pip from a local file. If you want to get all the dependencies of Twisted in a future-proof way, without network on production, just do this on a machine with the same arch/OS:

hawkowl /p/t/tx> pip download twisted[tls]==16.7.0rc1
[LOADING BARS ELINED]
Successfully downloaded twisted zope.interface constantly incremental Automat pyopenssl service-identity idna setuptools six characteristic cryptography pyasn1 pyasn1-modules attrs ipaddress enum34 cffi pycparser

hawkowl /p/t/tx> ls
Automat-0.3.0-py2.py3-none-any.whl ipaddress-1.0.17-py2-none-any.whl
Twisted-16.7.0rc1.tar.bz2 pyOpenSSL-16.2.0-py2.py3-none-any.whl
attrs-16.3.0-py2.py3-none-any.whl pyasn1-0.1.9-py2.py3-none-any.whl
cffi-1.9.1-cp27-cp27m-macosx_10_10_intel.whl pyasn1_modules-0.0.8-py2.py3-none-any.whl
characteristic-14.3.0-py2.py3-none-any.whl pycparser-2.17.tar.gz
constantly-15.1.0-py2.py3-none-any.whl service_identity-16.0.0-py2.py3-none-any.whl
cryptography-1.7.1-cp27-cp27m-macosx_10_10_intel.whl setuptools-32.3.0-py2.py3-none-any.whl
enum34-1.1.6-py2-none-any.whl six-1.10.0-py2.py3-none-any.whl
idna-2.2-py2.py3-none-any.whl zope.interface-4.3.3-cp27-cp27m-macosx_10_11_x86_64.whl
incremental-16.10.1-py2.py3-none-any.whl

Put these files on the target machine. Then on the target machine, with no internet connection:

hawkowl /p/t/tx> virtualenv venv
Using real prefix '/System/Library/Frameworks/Python.framework/Versions/2.7'
New python executable in /private/tmp/tx/venv/bin/python
Installing setuptools, pip, wheel...done.

# Because incremental is setup_requires on twisted, install it explicitly or setuptools will go try and fetch it
hawkowl /p/t/tx [1]> venv/bin/pip install --no-index --find-links ./ incremental
Collecting incremental
Installing collected packages: incremental
Successfully installed incremental-16.10.1

hawkowl /p/t/tx> venv/bin/pip install --no-index --find-links ./ twisted[tls]
Collecting twisted[tls]
Collecting Automat>=0.3.0 (from twisted[tls])
Collecting zope.interface>=3.6.0 (from twisted[tls])
Collecting constantly>=15.1 (from twisted[tls])
Requirement already satisfied: incremental>=16.10.1 in ./venv/lib/python2.7/site-packages (from twisted[tls])
Collecting service-identity; extra == "tls" (from twisted[tls])
Collecting idna>=0.6; extra == "tls" (from twisted[tls])
Collecting pyopenssl>=16.0.0; extra == "tls" (from twisted[tls])
Requirement already satisfied: six in ./venv/lib/python2.7/site-packages (from Automat>=0.3.0->twisted[tls])
Requirement already satisfied: characteristic in ./venv/lib/python2.7/site-packages (from Automat>=0.3.0->twisted[tls])
Requirement already satisfied: setuptools in ./venv/lib/python2.7/site-packages (from zope.interface>=3.6.0->twisted[tls])
Collecting pyasn1 (from service-identity; extra == "tls"->twisted[tls])
Collecting pyasn1-modules (from service-identity; extra == "tls"->twisted[tls])
Requirement already satisfied: attrs in ./venv/lib/python2.7/site-packages (from service-identity; extra == "tls"->twisted[tls])
Collecting cryptography>=1.3.4 (from pyopenssl>=16.0.0; extra == "tls"->twisted[tls])
Collecting ipaddress (from cryptography>=1.3.4->pyopenssl>=16.0.0; extra == "tls"->twisted[tls])
Collecting enum34 (from cryptography>=1.3.4->pyopenssl>=16.0.0; extra == "tls"->twisted[tls])
Collecting cffi>=1.4.1 (from cryptography>=1.3.4->pyopenssl>=16.0.0; extra == "tls"->twisted[tls])
Collecting pycparser (from cffi>=1.4.1->cryptography>=1.3.4->pyopenssl>=16.0.0; extra == "tls"->twisted[tls])
Installing collected packages: Automat, zope.interface, constantly, pyasn1, pyasn1-modules, idna, ipaddress, enum34, pycparser, cffi, cryptography, pyopenssl, service-identity, twisted
Successfully installed Automat-0.3.0 cffi-1.9.1 constantly-15.1.0 cryptography-1.7.1 enum34-1.1.6 idna-2.2 ipaddress-1.0.17 pyasn1-0.1.9 pyasn1-modules-0.0.8 pycparser-2.17 pyopenssl-16.2.0 service-identity-16.0.0 twisted-16.7.0rc1 zope.interface-4.3.3

This will install Twisted and its dependencies the official pip way, without needing to hardcode which libraries you fetch, and will do the right thing as Twisted eventually adds more dependencies (like when we split out FilePath and Deferred).

- Amber
Seems to require a new Python module, Automat. (Not mentioned in the NEWS file, INSTALL, etc.) I found Automat 0.3.0 (with Glyph's fingerprints on it) at PyPI, but it only exists in .whl format. Downloaded it, but I'm not sure what to do next. I was able to get it working by downloading and installing get-pip.py and Automat (contrary to the python.org documentation, I didn't have pip installed on my system even though it is Python 2.7.13 downloaded from python.org.)
sudo python get-pip.py
sudo pip install Automat
This is worrisome to me because I think this requires Internet access to pypi.python.org, which I have on my development and testing systems, but not on my production systems. Everything else I've ever needed, I could download and install source files.
Having done this, my app seems to run fine (only quick sanity testing so far.)
----install---
Searching for Automat>=0.3.0
Reading https://pypi.python.org/simple/Automat/ <https://pypi.python.org/simple/Automat/>
Download error on https://pypi.python.org/simple/Automat/ <https://pypi.python.org/simple/Automat/>: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590) -- Some packages may not be found!
Couldn't find index page for 'Automat' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/ <https://pypi.python.org/simple/>
Download error on https://pypi.python.org/simple/ <https://pypi.python.org/simple/>: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590) -- Some packages may not be found!
No local packages or working download links found for Automat>=0.3.0
error: Could not find suitable distribution for Requirement.parse('Automat>=0.3.0')
------
---try to use without Automat---
File "medusa_server.py", line 72, in <module>
from twisted.application import service, internet
File "/usr/lib/python2.7/site-packages/Twisted-16.7.0rc1-py2.7-linux-x86_64.egg/twisted/application/internet.py", line 54, in <module>
from automat import MethodicalMachine
ImportError: No module named automat
------
Hello everyone, and happy holidays!
- twisted.web.client.Agent now supports IPv6! It's also now the primary web client in Twisted, with twisted.web.client.getPage being deprecated in favour of it and Treq.
- twisted.internet.ssl.CertificateOptions has had its `method` argument deprecated, in favour of the new raiseMinimumTo, lowerMaximumSecurityTo, and insecurelyLowerMinimumTo arguments, which take TLSVersion arguments. This allows you to better give a range of versions of TLS you wish to negotiate, rather than forcing yourself to any one version.
- twisted.internet.ssl.CertificateOptions will use OpenSSL's MODE_RELEASE_BUFFERS, which will let it free unused memory that was held by idle TLS connections.
- You can now call the new twist runner with `python -m twisted`.
- Conch has the beginnings of ECDH key exchange, and supports `hmac-sha2-384` in twisted.conch.ssh.
- Better Unicode support in twisted.internet.reactor.spawnProcess, especially on Windows on Python 3.6.
- More Python 3 porting in Conch, and more under-the-hood changes to facilitate a Twisted-wide jump to new-style classes only on Python 2 in 2018/2019. This release has also been tested on Python 3.6 on Linux.
- Lots of deprecated code removals, to make a sleeker, less confusing Twisted.
- 20+ closed tickets.
python -m pip install Twisted==16.7.0rc1
Please test it, and let me know how your applications fare, good or bad! If nothing comes up, 16.7 looks like it'll land just before the new year.
With festive hoots,
Amber Brown
_______________________________________________
Twisted-Python mailing list
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python <http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python>
--
John Santos
Evans Griffiths & Hart, Inc.
781-861-0670 ext 539
_______________________________________________
Twisted-Python mailing list
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
John Santos
2016-12-29 17:43:22 UTC
Permalink
Thank you (and Glyph for his followup) for the detailed instructions. I
will be trying this on our 2nd development system in a few days (or
possibly sooner on the test system if 16.7.0 is released before I get
around to it :-)) and will let you know if I encounter any difficulties.
Post by Amber "Hawkie" Brown
Hi John,
.whl is just an optimised binary format, and can be installed by pip
from a local file. If you want to get all the dependencies of Twisted
in a future-proof way, without network on production, just do this on
hawkowl /p/t/tx> pip download twisted[tls]==16.7.0rc1
[LOADING BARS ELINED]
Successfully downloaded twisted zope.interface constantly incremental
Automat pyopenssl service-identity idna setuptools six characteristic
cryptography pyasn1 pyasn1-modules attrs ipaddress enum34 cffi pycparser
hawkowl /p/t/tx> ls
Automat-0.3.0-py2.py3-none-any.whl ipaddress-1.0.17-py2-none-any.whl
Twisted-16.7.0rc1.tar.bz2 pyOpenSSL-16.2.0-py2.py3-none-any.whl
attrs-16.3.0-py2.py3-none-any.whl pyasn1-0.1.9-py2.py3-none-any.whl
cffi-1.9.1-cp27-cp27m-macosx_10_10_intel.whl
pyasn1_modules-0.0.8-py2.py3-none-any.whl
characteristic-14.3.0-py2.py3-none-any.whl pycparser-2.17.tar.gz
constantly-15.1.0-py2.py3-none-any.whl
service_identity-16.0.0-py2.py3-none-any.whl
cryptography-1.7.1-cp27-cp27m-macosx_10_10_intel.whl
setuptools-32.3.0-py2.py3-none-any.whl
enum34-1.1.6-py2-none-any.whl six-1.10.0-py2.py3-none-any.whl
idna-2.2-py2.py3-none-any.whl
zope.interface-4.3.3-cp27-cp27m-macosx_10_11_x86_64.whl
incremental-16.10.1-py2.py3-none-any.whl
Put these files on the target machine. Then on the target machine,
hawkowl /p/t/tx> virtualenv venv
Using real prefix
'/System/Library/Frameworks/Python.framework/Versions/2.7'
New python executable in /private/tmp/tx/venv/bin/python
Installing setuptools, pip, wheel...done.
# Because incremental is setup_requires on twisted, install it
explicitly or setuptools will go try and fetch it
hawkowl /p/t/tx [1]> venv/bin/pip install --no-index --find-links ./ incremental
Collecting incremental
Installing collected packages: incremental
Successfully installed incremental-16.10.1
hawkowl /p/t/tx> venv/bin/pip install --no-index --find-links ./ twisted[tls]
Collecting twisted[tls]
Collecting Automat>=0.3.0 (from twisted[tls])
Collecting zope.interface>=3.6.0 (from twisted[tls])
Collecting constantly>=15.1 (from twisted[tls])
Requirement already satisfied: incremental>=16.10.1 in
./venv/lib/python2.7/site-packages (from twisted[tls])
Collecting service-identity; extra == "tls" (from twisted[tls])
Collecting idna>=0.6; extra == "tls" (from twisted[tls])
Collecting pyopenssl>=16.0.0; extra == "tls" (from twisted[tls])
Requirement already satisfied: six in
./venv/lib/python2.7/site-packages (from Automat>=0.3.0->twisted[tls])
Requirement already satisfied: characteristic in
./venv/lib/python2.7/site-packages (from Automat>=0.3.0->twisted[tls])
Requirement already satisfied: setuptools in
./venv/lib/python2.7/site-packages (from
zope.interface>=3.6.0->twisted[tls])
Collecting pyasn1 (from service-identity; extra == "tls"->twisted[tls])
Collecting pyasn1-modules (from service-identity; extra ==
"tls"->twisted[tls])
Requirement already satisfied: attrs in
./venv/lib/python2.7/site-packages (from service-identity; extra ==
"tls"->twisted[tls])
Collecting cryptography>=1.3.4 (from pyopenssl>=16.0.0; extra == "tls"->twisted[tls])
Collecting ipaddress (from cryptography>=1.3.4->pyopenssl>=16.0.0;
extra == "tls"->twisted[tls])
Collecting enum34 (from cryptography>=1.3.4->pyopenssl>=16.0.0; extra
== "tls"->twisted[tls])
Collecting cffi>=1.4.1 (from cryptography>=1.3.4->pyopenssl>=16.0.0;
extra == "tls"->twisted[tls])
Collecting pycparser (from
cffi>=1.4.1->cryptography>=1.3.4->pyopenssl>=16.0.0; extra ==
"tls"->twisted[tls])
Installing collected packages: Automat, zope.interface, constantly,
pyasn1, pyasn1-modules, idna, ipaddress, enum34, pycparser, cffi,
cryptography, pyopenssl, service-identity, twisted
Successfully installed Automat-0.3.0 cffi-1.9.1 constantly-15.1.0
cryptography-1.7.1 enum34-1.1.6 idna-2.2 ipaddress-1.0.17 pyasn1-0.1.9
pyasn1-modules-0.0.8 pycparser-2.17 pyopenssl-16.2.0
service-identity-16.0.0 twisted-16.7.0rc1 zope.interface-4.3.3
This will install Twisted and its dependencies the official pip way,
without needing to hardcode which libraries you fetch, and will do the
right thing as Twisted eventually adds more dependencies (like when we
split out FilePath and Deferred).
- Amber
Post by John Santos
Seems to require a new Python module, Automat. (Not mentioned in the
NEWS file, INSTALL, etc.) I found Automat 0.3.0 (with Glyph's
fingerprints on it) at PyPI, but it only exists in .whl format.
Downloaded it, but I'm not sure what to do next. I was able to get
it working by downloading and installing get-pip.py and Automat
(contrary to the python.org <http://python.org> documentation, I
didn't have pip installed on my system even though it is Python
2.7.13 downloaded from python.org <http://python.org>.)
sudo python get-pip.py
sudo pip install Automat
This is worrisome to me because I think this requires Internet access
to pypi.python.org <http://pypi.python.org>, which I have on my
development and testing systems, but not on my production systems.
Everything else I've ever needed, I could download and install source
files.
Having done this, my app seems to run fine (only quick sanity testing so far.)
----install---
Searching for Automat>=0.3.0
Reading https://pypi.python.org/simple/Automat/
CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590) --
Some packages may not be found!
Couldn't find index page for 'Automat' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590) --
Some packages may not be found!
No local packages or working download links found for Automat>=0.3.0
error: Could not find suitable distribution for
Requirement.parse('Automat>=0.3.0')
------
---try to use without Automat---
File "medusa_server.py", line 72, in <module>
from twisted.application import service, internet
File
"/usr/lib/python2.7/site-packages/Twisted-16.7.0rc1-py2.7-linux-x86_64.egg/twisted/application/internet.py",
line 54, in <module>
from automat import MethodicalMachine
ImportError: No module named automat
------
Hello everyone, and happy holidays!
Although it didn't quite fall on the solstice, one could call this
Twisted release a little ray of sunshine, with some nice new
- twisted.web.client.Agent now supports IPv6! It's also now the
primary web client in Twisted, with twisted.web.client.getPage being
deprecated in favour of it and Treq.
- twisted.internet.ssl.CertificateOptions has had its `method`
argument deprecated, in favour of the new raiseMinimumTo,
lowerMaximumSecurityTo, and insecurelyLowerMinimumTo arguments,
which take TLSVersion arguments. This allows you to better give a
range of versions of TLS you wish to negotiate, rather than forcing
yourself to any one version.
- twisted.internet.ssl.CertificateOptions will use OpenSSL's
MODE_RELEASE_BUFFERS, which will let it free unused memory that was
held by idle TLS connections.
- You can now call the new twist runner with `python -m twisted`.
- Conch has the beginnings of ECDH key exchange, and supports
`hmac-sha2-384` in twisted.conch.ssh.
- Better Unicode support in twisted.internet.reactor.spawnProcess,
especially on Windows on Python 3.6.
- More Python 3 porting in Conch, and more under-the-hood changes to
facilitate a Twisted-wide jump to new-style classes only on Python 2
in 2018/2019. This release has also been tested on Python 3.6 on Linux.
- Lots of deprecated code removals, to make a sleeker, less
confusing Twisted.
- 20+ closed tickets.
You can get the tarball and the NEWS file at
https://twistedmatrix.com/Releases/rc/16.7.0rc1/ , or you can try it
python -m pip install Twisted==16.7.0rc1
Please test it, and let me know how your applications fare, good or
bad! If nothing comes up, 16.7 looks like it'll land just before the
new year.
With festive hoots,
Amber Brown
_______________________________________________
Twisted-Python mailing list
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
--
John Santos
Evans Griffiths & Hart, Inc.
781-861-0670 ext 539
_______________________________________________
Twisted-Python mailing list
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
_______________________________________________
Twisted-Python mailing list
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
--
John Santos
Evans Griffiths & Hart, Inc.
781-861-0670 ext 539
Glyph Lefkowitz
2016-12-29 13:27:44 UTC
Permalink
Post by John Santos
Seems to require a new Python module, Automat.
Good catch, that probably should have been announced at least a little :).
Post by John Santos
(Not mentioned in the NEWS file, INSTALL, etc.)
Despite the fact that it would have been good to mention this particular dependency because its adoption is a new architectural direction for Twisted, in general we're actually planning to do less communication of dependencies in this way. Right now, we have redundant listing of dependencies in a few places, but in the future, setup.py (or more specifically, "packaging metadata") will be the one, authoritative place that such things are listed.
Post by John Santos
I found Automat 0.3.0 (with Glyph's fingerprints on it) at PyPI, but it only exists in .whl format.
Although you probably should just use the wheel, the fact that there was only a wheel was an oversight. I've uploaded an sdist and pushed a corresponding tag (after making sure that the source files all matched, of course).
Post by John Santos
Downloaded it, but I'm not sure what to do next. [...] This is worrisome to me because I think this requires Internet access to pypi.python.org, which I have on my development and testing systems, but not on my production systems. Everything else I've ever needed, I could download and install source files.
As Amber already explained, the ability to deploy offline is very much an important part of the pip/wheel ecosystem. I went into some detail in one of my own blog posts on this: https://glyph.twistedmatrix.com/2015/03/docker-deploy-double-dutch.html <https://glyph.twistedmatrix.com/2015/03/docker-deploy-double-dutch.html>

Increasingly, we are assuming a pip-like packaging toolchain for dealing with Twisted's dependencies, so getting familiar with this stuff - pip, requirements.txt, pinning, virtualenv - is worthwhile. (It'll make your life easier in more ways than one.) It should handily meet all your requirements for security and flexibility, and if it doesn't, we have a good working relationship with the maintainers of all those tools so we'd like to hear about if it's a problem :).
Post by John Santos
Having done this, my app seems to run fine (only quick sanity testing so far.)
Thanks for testing!

-glyph
Tim Allen
2016-12-30 00:20:05 UTC
Permalink
Post by Glyph Lefkowitz
Increasingly, we are assuming a pip-like packaging toolchain for
dealing with Twisted's dependencies, so getting familiar with this
stuff - pip, requirements.txt, pinning, virtualenv - is worthwhile.
(It'll make your life easier in more ways than one.)
I wasn't going to mention it (because we found a workaround), but since
the topic has come up...

My current employer has a fully pip-based packaging toolchain for all
our internal Python apps, where we build wheels for our apps and all
their dependencies, and then deploy them into a virtualenv in
production. For reproducability reasons, we have a PyPI mirror inside
the corporate firewall, and our wheel-building Docker image has
a `pip.conf` configured to point at it.

Twisted 16.6 broke this system when it added a package named
"incremental" to the `setup_requires` list in setup.py. Because
`setup_requires` is a feature provided by setuptools, packages listed
there are installed by (the same machinery as) setuptools' `ez_install`
command, which does *not* respect `pip.conf`, and so it tried to
download the package from upstream PyPI and timed out banging its head
against the firewall.

As a workaround, we messed with our build-scripts to manually "pip
install" incremental before installing Twisted, or anything that depends
on it, so everything's fine again.

The Python Packaging Authority already has plans for replacing
`setup_requires` with something more Pip-friendly (see PEP 518), so over
time this won't be an issue. For the short term, though, I hope there
won't be too many more things added to `setup_requires` without
announcement.


Tim.
Donald Stufft
2016-12-30 00:29:57 UTC
Permalink
Post by Tim Allen
Post by Glyph Lefkowitz
Increasingly, we are assuming a pip-like packaging toolchain for
dealing with Twisted's dependencies, so getting familiar with this
stuff - pip, requirements.txt, pinning, virtualenv - is worthwhile.
(It'll make your life easier in more ways than one.)
I wasn't going to mention it (because we found a workaround), but since
the topic has come up...
My current employer has a fully pip-based packaging toolchain for all
our internal Python apps, where we build wheels for our apps and all
their dependencies, and then deploy them into a virtualenv in
production. For reproducability reasons, we have a PyPI mirror inside
the corporate firewall, and our wheel-building Docker image has
a `pip.conf` configured to point at it.
Twisted 16.6 broke this system when it added a package named
"incremental" to the `setup_requires` list in setup.py. Because
`setup_requires` is a feature provided by setuptools, packages listed
there are installed by (the same machinery as) setuptools' `ez_install`
command, which does *not* respect `pip.conf`, and so it tried to
download the package from upstream PyPI and timed out banging its head
against the firewall.
As a workaround, we messed with our build-scripts to manually "pip
install" incremental before installing Twisted, or anything that depends
on it, so everything's fine again.
The Python Packaging Authority already has plans for replacing
`setup_requires` with something more Pip-friendly (see PEP 518), so over
time this won't be an issue. For the short term, though, I hope there
won't be too many more things added to `setup_requires` without
announcement.
FTR, you can configure setup_requires too, it just uses a different format.
In any of the files listed at https://docs.python.org/3.6/install/index.html#location-and-names-of-config-files <https://docs.python.org/3.6/install/index.html#location-and-names-of-config-files>
You can drop a

[easy_install]
Index_url = https://private.example.com/simple/
allow_hosts = private.example.com # not strictly required, but will error instead of timeout for non private.example.com

I’m not 100% sure on the exact syntax/structure, but something like that should work.


—
Donald Stufft
Glyph Lefkowitz
2016-12-30 02:07:32 UTC
Permalink
Post by Tim Allen
Post by Glyph Lefkowitz
Increasingly, we are assuming a pip-like packaging toolchain for
dealing with Twisted's dependencies, so getting familiar with this
stuff - pip, requirements.txt, pinning, virtualenv - is worthwhile.
(It'll make your life easier in more ways than one.)
I wasn't going to mention it (because we found a workaround), but since
the topic has come up...
My current employer has a fully pip-based packaging toolchain for all
our internal Python apps, where we build wheels for our apps and all
their dependencies, and then deploy them into a virtualenv in
production. For reproducability reasons, we have a PyPI mirror inside
the corporate firewall, and our wheel-building Docker image has
a `pip.conf` configured to point at it.
Twisted 16.6 broke this system when it added a package named
"incremental" to the `setup_requires` list in setup.py. Because
`setup_requires` is a feature provided by setuptools, packages listed
there are installed by (the same machinery as) setuptools' `ez_install`
command, which does *not* respect `pip.conf`, and so it tried to
download the package from upstream PyPI and timed out banging its head
against the firewall.
Thanks, this is very useful information.

If you wanted to have this verified for the future, a network-isolated buildbot would be a good way to ensure we don't mess this up in other ways. Based on this thread it seems that multiple users have this concern, so it's certainly not specific to you, either.

It would also force us to fix the dang documentation-generator test cases that keep failing when a certain someone forgets to update their SSL cert :-).
Post by Tim Allen
As a workaround, we messed with our build-scripts to manually "pip
install" incremental before installing Twisted, or anything that depends
on it, so everything's fine again.
The Python Packaging Authority already has plans for replacing
`setup_requires` with something more Pip-friendly (see PEP 518), so over
time this won't be an issue. For the short term, though, I hope there
won't be too many more things added to `setup_requires` without
announcement.
Incremental is a bit of a special case, since "version definition" is one of the few things that needs synchronization between setup-time and run-time. I was pretty nervous about adding `setup_requires`, but the value of getting our versioning logic outside of Twisted itself seemed worth the potential packaging fallout.

So, while you can expect to see many more things showing up in install_requires unannounced, as we split out low-level portions of Twisted and adopt libraries like h11, I doubt that you'll see another setup_requires arrive, especially before PyPA gets together a better approach than the literal setup.py setup_requires. Given that you've specifically asked for notice, any project member submitting or reviewing a patch that adds one should give a shout here.

-glyph
Terekhov, Mikhail
2017-01-03 15:58:43 UTC
Permalink
Post by Glyph Lefkowitz
Post by John Santos
Seems to require a new Python module, Automat.
Good catch, that probably should have been announced at least a little :).
Post by John Santos
(Not mentioned in the NEWS file, INSTALL, etc.)
Despite the fact that it would have been good to mention this
particular dependency because its adoption is a new architectural
direction for Twisted, in /general/ we're actually planning to do less
communication of dependencies in this way. Right now, we have
redundant listing of dependencies in a few places, but in the future,
setup.py (or more specifically, "packaging metadata") will be the one,
authoritative place that such things are listed.
Post by John Santos
I found Automat 0.3.0 (with Glyph's fingerprints on it) at PyPI, but
it only exists in .whl format.
Although you probably /should/ just use the wheel, the fact that there
was /only/ a wheel was an oversight. I've uploaded an sdist and
pushed a corresponding tag (after making sure that the source files
all matched, of course).
It is probably an off topic but it would not hurt to add a more verbose
description and a link to the sources on PyPI for the Automat package
especially if it is so important for Twisted.
Glyph Lefkowitz
2017-01-03 20:17:33 UTC
Permalink
Post by Glyph Lefkowitz
Post by John Santos
Seems to require a new Python module, Automat.
Good catch, that probably should have been announced at least a little :).
Post by John Santos
(Not mentioned in the NEWS file, INSTALL, etc.)
Despite the fact that it would have been good to mention this particular dependency because its adoption is a new architectural direction for Twisted, in general we're actually planning to do less communication of dependencies in this way. Right now, we have redundant listing of dependencies in a few places, but in the future, setup.py (or more specifically, "packaging metadata") will be the one, authoritative place that such things are listed.
Post by John Santos
I found Automat 0.3.0 (with Glyph's fingerprints on it) at PyPI, but it only exists in .whl format.
Although you probably should just use the wheel, the fact that there was only a wheel was an oversight. I've uploaded an sdist and pushed a corresponding tag (after making sure that the source files all matched, of course).
It is probably an off topic but it would not hurt to add a more verbose description and a link to the sources on PyPI for the Automat package especially if it is so important for Twisted.
Please feel free to submit a PR to automat to do this. There's lots of descriptive material here: <https://github.com/glyph/automat/blob/master/README.md <https://github.com/glyph/automat/blob/master/README.md>> which you could use to update the PyPI metadata part.

-glyph

Continue reading on narkive:
Search results for '[Twisted-Python] Twisted 16.7.0rc1 Release Candidate Announcement' (Questions and Answers)
19
replies
what do u think abt mk gandhi??????
started 2009-12-02 04:19:42 UTC
politics
Loading...