[Twisted-Python] Twisted 17.1 Release Announcement

[Amber Hawkie Brown]

On behalf of Twisted Matrix Laboratories, I am honoured to announce the release of Twisted 17.1!

The highlights of this release are:

- twisted.web.client.Agent now supports IPv6! It's also now the primary web client in Twisted, with twisted.web.client.getPage being deprecated in favour of it and Treq.
- twisted.web.server has had many cleanups revolving around timing out inactive clients.
- twisted.internet.ssl.CertificateOptions has had its `method` argument deprecated, in favour of the new raiseMinimumTo, lowerMaximumSecurityTo, and insecurelyLowerMinimumTo arguments, which take TLSVersion arguments. This allows you to better give a range of versions of TLS you wish to negotiate, rather than forcing yourself to any one version.
- twisted.internet.ssl.CertificateOptions will use OpenSSL's MODE_RELEASE_BUFFERS, which will let it free unused memory that was held by idle TLS connections.
- You can now call the new twist runner with `python -m twisted`.
- twisted.conch.ssh now has some ECDH key exchange support and supports `hmac-sha2-384`.
- Better Unicode support in twisted.internet.reactor.spawnProcess, especially on Windows on Python 3.6.
- More Python 3 porting in Conch, and more under-the-hood changes to facilitate a Twisted-wide jump to new-style classes only on Python 2 in 2018/2019. This release has also been tested on Python 3.6 on Linux.
- Lots of deprecated code removals, to make a sleeker, less confusing Twisted.
- 60+ closed tickets.

For more information, check the NEWS file (link provided below).

You can find the downloads at <https://pypi.python.org/pypi/Twisted <https://pypi.python.org/pypi/Twisted>> (or alternatively <http://twistedmatrix.com/trac/wiki/Downloads <http://twistedmatrix.com/trac/wiki/Downloads>>). The NEWS file is also available at <https://github.com/twisted/twisted/blob/twisted-17.1.0/NEWS <https://github.com/twisted/twisted/blob/twisted-17.1.0/NEWS>>.

Many thanks to everyone who had a part in this release - the supporters of the Twisted Software Foundation, the developers who contributed code as well as documentation, and all the people building great things with Twisted!

Twisted Regards,
Amber Brown (HawkOwl)

[Tristan Seligmann]

Post by Amber Hawkie Brown
- Lots of deprecated code removals, to make a sleeker, less confusing Twisted.

Something perhaps not obvious from the release notes:

String endpoint descriptions, as used by twist/twistd among other things,
for listening on tcp ports now require the "tcp:": that is, "80" is no
longer accepted, you need to use "tcp:80".

This functionality has been deprecated for a while, but as it is unlikely
that many people run twist/twistd with DeprecationWarnings enabled, most
people relying on this behaviour have unfortunately probably never seen the
warning.

[Phil Mayers]

Post by Tristan Seligmann
String endpoint descriptions, as used by twist/twistd among other
things, for listening on tcp ports now require the "tcp:": that is, "80"
is no longer accepted, you need to use "tcp:80".

In which case there's a regression in the "portforward" twistd plugin:

$ v/bin/twistd portforward
Traceback (most recent call last):
...
ValueError: Unknown endpoint type: '6666'

$ v/bin/twistd portforward -p tcp:6666 -d tcp:6665
...
ValueError: invalid literal for int() with base 10: 'tcp:6665'

:o(

[Phil Mayers]

Post by Tristan Seligmann
String endpoint descriptions, as used by twist/twistd among other
things, for listening on tcp ports now require the "tcp:": that is, "80"
is no longer accepted, you need to use "tcp:80".

Actually this works:

twistd portforward -p tcp:6666 -d 6665

Odd but meh, I'll take it

[Glyph Lefkowitz]

Post by Phil Mayers

Post by Tristan Seligmann
String endpoint descriptions, as used by twist/twistd among other
things, for listening on tcp ports now require the "tcp:": that is, "80"
is no longer accepted, you need to use "tcp:80".

twistd portforward -p tcp:6666 -d 6665
Odd but meh, I'll take it

Yeah, this is a lack of test coverage for the 'default' case. Technically it is a regression, but it's probably pretty unusual to use the default; point being, I don't think we need an emergency release here.

There's also a huge problem here with documentation, where `-dÂŽ is a TCP port but `-pÂŽ is a strport, and it's hard for end-users to get from the command line to the relevant documentation that explains what strports are available.

Nevertheless - please file a ticket!

-glyph

[Phil Mayers]

Post by Glyph Lefkowitz
Yeah, this is a lack of test coverage for the 'default' case.
Technically it /is/ a regression, but it's probably pretty unusual to
use the default; point being, I don't think we need an emergency release
here.

Agreed. It's completely tolerable.

Post by Glyph Lefkowitz
Nevertheless - please file a ticket!

#9033 done

[Glyph Lefkowitz]

Post by Phil Mayers

Post by Glyph Lefkowitz
Yeah, this is a lack of test coverage for the 'default' case.
Technically it /is/ a regression, but it's probably pretty unusual to
use the default; point being, I don't think we need an emergency release
here.

Agreed. It's completely tolerable.

Post by Glyph Lefkowitz
Nevertheless - please file a ticket!

#9033 done

Thanks for filing this!

-glyph

[Tristan Seligmann]

Post by Glyph Lefkowitz
Yeah, this is a lack of test coverage for the 'default' case. Technically
it *is* a regression, but it's probably pretty unusual to use the
default; point being, I don't think we need an emergency release here.

The functionality that was removed was previously deprecated, so I don't
think it counts as a regression. However, the fact that most people
wouldn't ever have seen the DeprecationWarning, and


There's also a huge problem here with documentation, where `-dÂŽ is a TCP

Post by Glyph Lefkowitz
port but `-pÂŽ is a strport, and it's hard for end-users to get from the
command line to the relevant documentation that explains what strports are
available.


the documentation issue does make things more awkward than they needed to
be.

[Glyph Lefkowitz]

Post by Glyph Lefkowitz
Yeah, this is a lack of test coverage for the 'default' case. Technically it is a regression, but it's probably pretty unusual to use the default; point being, I don't think we need an emergency release here.
The functionality that was removed was previously deprecated, so I don't think it counts as a regression.

The regression is that the default is wrong; we're effectively the one "calling" the deprecated API by supplying that data.

The fact that users won't see this in this context suggests that our deprecation process for user-facing command-line arguments is broken; we need to fix this for future iterations.

Post by Glyph Lefkowitz
However, the fact that most people wouldn't ever have seen the DeprecationWarning, and

There's also a huge problem here with documentation, where `-dÂŽ is a TCP port but `-pÂŽ is a strport, and it's hard for end-users to get from the command line to the relevant documentation that explains what strports are available.

the documentation issue does make things more awkward than they needed to be.

I think the documentation issue may require code changes to be addressed. Particularly, a command-line interface to say "hey, what endpoint description plugins are available, and what do they do" would be very helpful to packages like txtorcon and txacme.

-glyph

[steven meiers]

hi,


there is a abvious error in the code at the end.
since i know next to nothing about threads i thought maybe a print
statement in the code that is actually doing the work would give me
some pointers.

if im not mistaken, it is:
def callWithContext in python/context.py


as it turns out it does output nothing so i went back the call chain
until it prints something.
now im in:
internet/threads.py def deferToThreadPool

but even there, printing the argument f prints nothing?
isnt that the first method that is called from my code?

im on linux with python 3.x and twisted 17





from twisted.internet import reactor, task, threads
import datetime

def aSillyBlockingMethod(x):
    import time
    time.sleep(4)
    return x


def runEverySecond():
    print('x')
    
    print(datetime.now().strftime('%H:%M:%S'))    #WRONG, ERROR:
AttributeError: 'module' object has no attribute 'now'
    # that error gets eaten by twisted....why?
    #print(datetime.datetime.now().strftime('%H:%M:%S'))    #WORKS


def printResult(result):
    print("printResult:", result)

def printError(failure):
    print(failure)


# this will "sleep" for x seconds
d = threads.deferToThread(aSillyBlockingMethod, 'some argument')
d.addCallback(printResult)
d.addErrback(printError)

# but this will still run, not getting blocked
l = task.LoopingCall(runEverySecond)
l.start(1.0)

reactor.run()

[L. Daniel Burr]

Hi Steven,

On February 13, 2017 at 9:24:03 AM, steven meiers (***@yahoo.de) wrote:

hi,


there is a abvious error in the code at the end.
since i know next to nothing about threads i thought maybe a print
statement in the code that is actually doing the work would give me
some pointers.

if im not mistaken, it is:
def callWithContext in python/context.py


as it turns out it does output nothing so i went back the call chain
until it prints something.
now im in:
internet/threads.py def deferToThreadPool

but even there, printing the argument f prints nothing?
isnt that the first method that is called from my code?


Sorry, your code example and your explanation don’t make clear what you expect to happen, and when.  When you say “the first method that is called from my code”, are you referring to  aSillyBlockingMethod, or runEverySecond?  In the deferToThread case, you are calling the former; in the task.LoopingCall case, you are calling the latter.

If your intent is to trigger the strftime exception as a consequence of your call to deferToThread, then you would need to either add runEverySecond to the callback chain, or move the call to strftime into the aSillyBlockingMethod call.

Example:

# this will "sleep" for x seconds
d = threads.deferToThread(aSillyBlockingMethod, 'some argument’)
d.addCallback(runEverySecond)
d.addCallback(printResult)
d.addErrback(printError)

Hope this helps,

L. Daniel Burr

[steven meiers]

Sorry, your code example and your explanation don’t make clear what
you expect to happen, and when.  When you say “the first method that
is called from my code”, are you referring to  aSillyBlockingMethod,
or runEverySecond?  In the deferToThread case, you are calling the
former; in the task.LoopingCall case, you are calling the latter.

Ah youre right, i forgot to write about the most important part.
if the datetime line is wrong you get
Unhandled Error

i was looking for a way to get the original error message that:
print(datetime.now().strftime('%H:%M:%S'))

throws if used outside of twisted code.

i know how to correct the code but i would like to know why i dont get
that error on the console.

If your intent is to trigger the strftime exception as a consequence
of your call to deferToThread, then you would need to either add
runEverySecond to the callback chain, or move the call to strftime
into the aSillyBlockingMethod call.
# this will "sleep" for x seconds
d = threads.deferToThread(aSillyBlockingMethod, 'some argument’)
d.addCallback(runEverySecond)
d.addCallback(printResult)
d.addErrback(printError)

i tried that one, code looks like this. but i dont get the error
message...still Unhandled error in Deferred

from twisted.internet import reactor, task, threads
import datetime

def aSillyBlockingMethod(x):
    import time
    time.sleep(4)
    return x


def runEverySecond(ignored):

    print(datetime.now().strftime('%H:%M:%S'))    #WRONG, ERROR:
AttributeError: 'module' object has no attribute 'now'
    # that error gets eaten by twisted....why?
    #print(datetime.datetime.now().strftime('%H:%M:%S'))    #WORKS


def printResult(result):
    print("printResult:", result)

def printError(failure):
    print(failure)


# this will "sleep" for x seconds
d = threads.deferToThread(aSillyBlockingMethod, 'some argument')
d.addCallback(runEverySecond)
d.addCallback(printResult)
d.addErrback(printError)

# but this will still run, not getting blocked
l = task.LoopingCall(runEverySecond, 'lala')
l.start(1.0)

reactor.callLater(5, reactor.stop)
reactor.run()

[L. Daniel Burr]

Hi Steven,

Post by L. Daniel Burr
Sorry, your code example and your explanation don’t make clear what
you expect to happen, and when.  When you say “the first method that
is called from my code”, are you referring to  aSillyBlockingMethod,
or runEverySecond?  In the deferToThread case, you are calling the
former; in the task.LoopingCall case, you are calling the latter.

Ah youre right, i forgot to write about the most important part.
if the datetime line is wrong you get
Unhandled Error

i was looking for a way to get the original error message that:
print(datetime.now().strftime('%H:%M:%S'))

throws if used outside of twisted code.

i know how to correct the code but i would like to know why i dont get
that error on the console.


[SNIP]

The “Unhanded error in Deferred” isn’t coming from your call to deferToThread, it is coming from your task.LoopingCall.  When you invoke .start() on your LoopingCall instance, it returns a Deferred, which fires its assigned callback handler when you call .stop() on the LoopingCall instance, and which fires its assigned errback handler if an exception occurs while running the function (in this case runEverySecond) passed to the LoopingCall.

In other words, you need this:

l = task.LoopingCall(runEverySecond, ‘lala’)
l.addErrback(printError)

Hope this helps,

L. Daniel Burr

[steven meiers]

Post by L. Daniel Burr
[SNIP]
The “Unhanded error in Deferred” isn’t coming from your call to
deferToThread, it is coming from your task.LoopingCall.  When you
invoke .start() on your LoopingCall instance, it returns a Deferred,
which fires its assigned callback handler when you call .stop() on
the LoopingCall instance, and which fires its assigned errback
handler if an exception occurs while running the function (in this
case runEverySecond) passed to the LoopingCall.
l = task.LoopingCall(runEverySecond, ‘lala’)
l.addErrback(printError)

you probably meant:
l = task.LoopingCall(runEverySecond, 'lala')
d = l.start(1.0)
d.addErrback(printError)


your version gives me a attribute error, thanks for clarifying that.

[Jean-Paul Calderone]

On Sat, Feb 11, 2017 at 5:03 AM, Amber Hawkie Brown <

Post by Amber Hawkie Brown
On behalf of Twisted Matrix Laboratories, I am honoured to announce the
release of Twisted 17.1!
- twisted.web.client.Agent now supports IPv6! It's also now the primary
web client in Twisted, with twisted.web.client.getPage being deprecated in
favour of it and Treq.
- twisted.web.server has had many cleanups revolving around timing out inactive clients.
- twisted.internet.ssl.CertificateOptions has had its `method` argument
deprecated, in favour of the new raiseMinimumTo, lowerMaximumSecurityTo,
and insecurelyLowerMinimumTo arguments, which take TLSVersion arguments.
This allows you to better give a range of versions of TLS you wish to
negotiate, rather than forcing yourself to any one version.
- twisted.internet.ssl.CertificateOptions will use OpenSSL's
MODE_RELEASE_BUFFERS, which will let it free unused memory that was held by
idle TLS connections.
- You can now call the new twist runner with `python -m twisted`.
- twisted.conch.ssh now has some ECDH key exchange support and supports `hmac-sha2-384`.
- Better Unicode support in twisted.internet.reactor.spawnProcess,
especially on Windows on Python 3.6.
- More Python 3 porting in Conch, and more under-the-hood changes to
facilitate a Twisted-wide jump to new-style classes only on Python 2 in
2018/2019. This release has also been tested on Python 3.6 on Linux.
- Lots of deprecated code removals, to make a sleeker, less confusing Twisted.
- 60+ closed tickets.
For more information, check the NEWS file (link provided below).
You can find the downloads at <https://pypi.python.org/pypi/Twisted> (or
alternatively <http://twistedmatrix.com/trac/wiki/Downloads>). The NEWS
file is also available at <https://github.com/twisted/
twisted/blob/twisted-17.1.0/NEWS>.
Many thanks to everyone who had a part in this release - the supporters of
the Twisted Software Foundation, the developers who contributed code as
well as documentation, and all the people building great things with
Twisted!

Thanks all! Great to see the troubles of 16.7 mostly behind us.

I do hope there will be at 17.1.1 soon for <
https://twistedmatrix.com/trac/ticket/9031> and particularly <
https://twistedmatrix.com/trac/ticket/9032>.

Jean-Paul

[Glyph Lefkowitz]

Post by Amber Hawkie Brown
On behalf of Twisted Matrix Laboratories, I am honoured to announce the release of Twisted 17.1!
- twisted.web.client.Agent now supports IPv6! It's also now the primary web client in Twisted, with twisted.web.client.getPage being deprecated in favour of it and Treq.
- twisted.web.server has had many cleanups revolving around timing out inactive clients.
- twisted.internet.ssl.CertificateOptions has had its `method` argument deprecated, in favour of the new raiseMinimumTo, lowerMaximumSecurityTo, and insecurelyLowerMinimumTo arguments, which take TLSVersion arguments. This allows you to better give a range of versions of TLS you wish to negotiate, rather than forcing yourself to any one version.
- twisted.internet.ssl.CertificateOptions will use OpenSSL's MODE_RELEASE_BUFFERS, which will let it free unused memory that was held by idle TLS connections.
- You can now call the new twist runner with `python -m twisted`.
- twisted.conch.ssh now has some ECDH key exchange support and supports `hmac-sha2-384`.
- Better Unicode support in twisted.internet.reactor.spawnProcess, especially on Windows on Python 3.6.
- More Python 3 porting in Conch, and more under-the-hood changes to facilitate a Twisted-wide jump to new-style classes only on Python 2 in 2018/2019. This release has also been tested on Python 3.6 on Linux.
- Lots of deprecated code removals, to make a sleeker, less confusing Twisted.
- 60+ closed tickets.
For more information, check the NEWS file (link provided below).
You can find the downloads at <https://pypi.python.org/pypi/Twisted <https://pypi.python.org/pypi/Twisted>> (or alternatively <http://twistedmatrix.com/trac/wiki/Downloads <http://twistedmatrix.com/trac/wiki/Downloads>>). The NEWS file is also available at <https://github.com/twisted/twisted/blob/twisted-17.1.0/NEWS <https://github.com/twisted/twisted/blob/twisted-17.1.0/NEWS>>.
Many thanks to everyone who had a part in this release - the supporters of the Twisted Software Foundation, the developers who contributed code as well as documentation, and all the people building great things with Twisted!
Thanks all! Great to see the troubles of 16.7 mostly behind us.

Yes, thanks! Lots of good stuff in this release!

Post by Amber Hawkie Brown
I do hope there will be at 17.1.1 soon for <https://twistedmatrix.com/trac/ticket/9031 <https://twistedmatrix.com/trac/ticket/9031>> and particularly <https://twistedmatrix.com/trac/ticket/9032 <https://twistedmatrix.com/trac/ticket/9032>>.

This is a little outside the normal process, but I added a milestone for these just so we have a thing to track when they're fixed and we should roll them up (and in case anything new appears):

https://twistedmatrix.com/trac/milestone/17.1.1 <https://twistedmatrix.com/trac/milestone/17.1.1>

Something I should note about these regressions; they showed up as soon as Mimic upgraded its dependency: https://github.com/rackerlabs/mimic/pull/734 <https://github.com/rackerlabs/mimic/pull/734>

This suggests to me that we really ought to have a more "active" phase of the release process where someone - let's say, the reviewer of the release ticket - goes out to check a select list of other projects' smoke tests to ensure that the prerelease is at least passing their CI. For example, I would normally have tested Mimic myself, but I was out of commission with a nasty flu-like thing for a good chunk of the prerelease period, but since I happened to miss that window, well-known projects (heck, treq has its own tests for .testing, and those broke, and it's in the twisted org!) managed to break without getting noticed.

So, while it would be good if user projects reported breakages, we should be checking at least the most obvious parts of the integration ecosystem ourselves as part of the release process.

-glyph

[Jean-Paul Calderone]

On Feb 11, 2017, at 5:54 PM, Jean-Paul Calderone <
On Sat, Feb 11, 2017 at 5:03 AM, Amber Hawkie Brown <

Post by Amber Hawkie Brown
On behalf of Twisted Matrix Laboratories, I am honoured to announce the
release of Twisted 17.1!
- twisted.web.client.Agent now supports IPv6! It's also now the primary
web client in Twisted, with twisted.web.client.getPage being deprecated in
favour of it and Treq.
- twisted.web.server has had many cleanups revolving around timing out inactive clients.
- twisted.internet.ssl.CertificateOptions has had its `method` argument
deprecated, in favour of the new raiseMinimumTo, lowerMaximumSecurityTo,
and insecurelyLowerMinimumTo arguments, which take TLSVersion arguments.
This allows you to better give a range of versions of TLS you wish to
negotiate, rather than forcing yourself to any one version.
- twisted.internet.ssl.CertificateOptions will use OpenSSL's
MODE_RELEASE_BUFFERS, which will let it free unused memory that was held by
idle TLS connections.
- You can now call the new twist runner with `python -m twisted`.
- twisted.conch.ssh now has some ECDH key exchange support and supports `hmac-sha2-384`.
- Better Unicode support in twisted.internet.reactor.spawnProcess,
especially on Windows on Python 3.6.
- More Python 3 porting in Conch, and more under-the-hood changes to
facilitate a Twisted-wide jump to new-style classes only on Python 2 in
2018/2019. This release has also been tested on Python 3.6 on Linux.
- Lots of deprecated code removals, to make a sleeker, less confusing Twisted.
- 60+ closed tickets.
For more information, check the NEWS file (link provided below).
You can find the downloads at <https://pypi.python.org/pypi/Twisted> (or
alternatively <http://twistedmatrix.com/trac/wiki/Downloads>). The NEWS
file is also available at <https://github.com/twisted/tw
isted/blob/twisted-17.1.0/NEWS>.
Many thanks to everyone who had a part in this release - the supporters
of the Twisted Software Foundation, the developers who contributed code as
well as documentation, and all the people building great things with
Twisted!

Thanks all! Great to see the troubles of 16.7 mostly behind us.
Yes, thanks! Lots of good stuff in this release!
I do hope there will be at 17.1.1 soon for <https://twistedmatrix.com/
trac/ticket/9031> and particularly <https://twistedmatrix.com/
trac/ticket/9032>.
This is a little outside the normal process, but I added a milestone for
these just so we have a thing to track when they're fixed and we should
https://twistedmatrix.com/trac/milestone/17.1.1
Something I should note about these regressions; they showed up as soon as
Mimic upgraded its dependency: https://github.
com/rackerlabs/mimic/pull/734
This suggests to me that we really ought to have a more "active" phase of
the release process where someone - let's say, the reviewer of the release
ticket - goes out to check a select list of other projects' smoke tests to
ensure that the prerelease is at least passing their CI. For example, I
would normally have tested Mimic myself, but I was out of commission with a
nasty flu-like thing for a good chunk of the prerelease period, but since I
happened to miss that window, well-known projects (heck, treq has its own
tests for .testing, and those broke, and it's in the twisted org!) managed
to break without getting noticed.

I wonder about the details of how treq's failures went unnoticed. Is
development sufficiently inactive that no one looked at CI between the
breakage and the release? Wasn't that a period of several months? Or does
it have a Twisted ***@HEAD build that none of the contributors really pay
attention to?

If the problem is attention, maybe some kind of alerting could be set up so
the folks who are actually working on Twisted have a direct feed of
information that might be related to that work creating regressions? I
know there are problems with false positives from CI but the dial for
information flow seems to be set a bit too low right now. Maybe turning it
up for a while and then re-evaluating makes sense.

So, while it would be good if user projects reported breakages, we should
be checking at least the *most* obvious parts of the integration
ecosystem ourselves as part of the release process.

All other things being equal, it would be better to learn about the
breakage at the time of the breakage rather than at release time (for the
usual reasons). So I think something like a mailing list for external CI
failures against Twisted ***@HEAD is a better thing to try than telling
the RM to go look at a bunch of CIs.

Maybe we could even double down on this and host the new list solely on
txlists? :)

Jean-Paul

[Tristan Seligmann]

Post by Jean-Paul Calderone
I wonder about the details of how treq's failures went unnoticed. Is
development sufficiently inactive that no one looked at CI between the
breakage and the release? Wasn't that a period of several months? Or does
pay attention to?

The latter is the case; there are Twisted trunk builds, but they're flagged
as "failures allowed" to avoid disrupting development with transient
breakage. Unfortunately this means that basically nobody ever notices when
they fail. For example, here's a failing build:

https://travis-ci.org/twisted/treq/builds/194691849

I haven't found a good solution to this problem yet in my own projects; I'm
hoping someone else has some ideas?

[Jean-Paul Calderone]

On Wed, 15 Feb 2017 at 15:23 Jean-Paul Calderone <

Post by Jean-Paul Calderone
I wonder about the details of how treq's failures went unnoticed. Is
development sufficiently inactive that no one looked at CI between the
breakage and the release? Wasn't that a period of several months? Or does
pay attention to?

The latter is the case; there are Twisted trunk builds, but they're
flagged as "failures allowed" to avoid disrupting development with
transient breakage. Unfortunately this means that basically nobody ever
https://travis-ci.org/twisted/treq/builds/194691849
I haven't found a good solution to this problem yet in my own projects;
I'm hoping someone else has some ideas?

Given an arbitrarily powerful CI system, you could do something.

Assume two projects, A and B, with a dependency between them (doesn't
matter which direction, I think; doesn't even have to be one-way though
obviously it should be).

When A changes, do a build of B. Use the new A and the newest version of B
which had a successful build. If this build succeeds, A and B are probably
still working well together. If this build fails, there's a good chance A
just introduced a regression.

Apply the same logic with A and B reversed.

I expect you could implement this on BuildBot. It might be more realistic
to do something slightly simpler - like pinning the version of B in the
A-changed builds (and vice versa). This would miss regressions relating to
changes in B introduced after the pinned version - but maybe that's okay
(you could update pinned versions with each release of B or once every few
months or something). I think you could implement this on something like
travis-ci without *much* difficulty. Just duplicate (some of) the treq
(for example) travis-ci configuration in Twisted's travis-ci
configuration. It doesn't sound crazy to say "treq 16.12.0 test suite
should be required to pass for Twisted ***@HEAD (at least until around
Twisted 17.12, probably?).

And of course there's bound to be noise out of this system, too. I'd guess
it would be little enough that the result is still valuable but hard to say
for sure without trying...

Jean-Paul

[Glyph Lefkowitz]

That seems like a reasonable requirement, yes.

-g

[Jean-Paul Calderone]

On Feb 15, 2017, at 6:51 AM, Jean-Paul Calderone <
It doesn't sound crazy to say "treq 16.12.0 test suite should be required
probably?).
That seems like a reasonable requirement, yes.

Here's *a* take on that: https://tm.tl/#9042 /
https://github.com/twisted/twisted/pull/710

I didn't pin treq to 16.12.0 but instead let it float on the most recent
release. That seems safe as long as treq takes care not to issue a release
that contains changes that makes it incompatible with Twisted ***@HEAD.
treq CI tests against Twisted ***@HEAD so I think this is low risk.

Jean-Paul

[Glyph Lefkowitz]

Post by Glyph Lefkowitz
That seems like a reasonable requirement, yes.
Here's a take on that: https://tm.tl/#9042 <https://tm.tl/#9042> / https://github.com/twisted/twisted/pull/710 <https://github.com/twisted/twisted/pull/710>

Yes. This sounds good to me. We might want to add one or two other downstream projects as well (starting with Klein), but Treq is a really good start. Perhaps we should ask other projects (Scrapy? Matrix?) to send us PRs to add some subset of their tests to this smoke test.

Frankly if this integrated a fix for the MemoryReactor behavior and was therefore passing I'd just land it :). Do you want to make it allow_fail?

-glyph

[Glyph Lefkowitz]

https://travis-ci.org/twisted/treq/builds/194691849 <https://travis-ci.org/twisted/treq/builds/194691849>
I haven't found a good solution to this problem yet in my own projects; I'm hoping someone else has some ideas?

Indeed, I probably saw these failures myself; the issue is that there's no point in the process where someone has to care. When you're trying to merge a PR, you care that the tests are failing for that PR, because making them pass is a part of the process. But, other stuff that's changed upstream is ancillary, even if you sort of abstractly know it's a problem. I suspect even our release manager, who also maintains treq on occasion, might have seen the failures at some point, but not managed to connect them.

If we could get requires.io <http://requires.io/> to send "--pre" requirements PRs, that might be enough; I notice immediately when dependencies actually break on production PyPI for Mimic, but there's no PR for prerelease dependency versions.

-glyph