[Twisted-Python] Waiting for transports to close

[Chris Norman]

Hi all,
I'm writing a MUD server, and I want a way for transports to be notified
ofa shutdown before being disconnected, and the reactor being stopped.

I've tried:

for t in transports:
t.write('Shutting down.\r\n')
t.loseConnection()
reactor.stop()

This doesn't seem to notify the transports.

I also tried:
for t in transports:
t.write('Shutting down.\r\n')
t.loseConnection()
while t.connected:
pass
reactor.stop()

That just blocked and did nothing, presumably something do with my while
loop.

Is there a stopWhenEmpty function on the somewhere? I did look over the
methods, and I couldn't find anything promising.

I'm just using the standard from twisted.internet import reactor
reactor, so no special cases here. In case it matters the transports I'm
using are twisted.protocols.basic.LineReceiver, and everything else
works with them.

Cheers in advance for the help.

[Glyph Lefkowitz]

Post by Chris Norman
Hi all,
I'm writing a MUD server, and I want a way for transports to be notified ofa shutdown before being disconnected, and the reactor being stopped.
t.write('Shutting down.\r\n')
t.loseConnection()
reactor.stop()
This doesn't seem to notify the transports.
t.write('Shutting down.\r\n')
t.loseConnection()
pass
reactor.stop()
That just blocked and did nothing, presumably something do with my while loop.
Is there a stopWhenEmpty function on the somewhere? I did look over the methods, and I couldn't find anything promising.
I'm just using the standard from twisted.internet import reactor reactor, so no special cases here. In case it matters the transports I'm using are twisted.protocols.basic.LineReceiver, and everything else works with them.
Cheers in advance for the help.

This is definitely doable, but before I explain it would help to know why you want to do this.

The reason I ask is: servers crash; hardware fails. The falcon cannot hear the falconer; things fall apart; the centre cannot hold.

When those servers do crash (and they will), you don't get a clean notification of disconnects. So if you're writing your application to rely very heavily on the ability to do a clean shutdown and get notifications of every disconnect at the time you expect reactor.stop() to be running, you are probably designing your system in a way that will be very fragile and prone to data loss. I know, I have made this mistake more than once myself :).

So, before you continue: do you actually need to do this? Could you just ignore the notification of the connection drop and exit gracefully, perhaps properly cleaning up whatever state is left over at next startup? If you really need this, understanding why you need it would also help in determining which implementation technique to suggest (there are a few).

-glyph

[Chris Norman]

Hi,
It's a MUD server, so players type in commands and receive textual
responses.

One of the admin commands is the ability to shutdown the server (or
CTRL-C might be pressed on the console). I'd like this action to notify
all connected transports that the server is going down for shutdown, so
they're not rudely disconnected, then once the notifications have all
gone through, then the server is free to shutdown.

I hope all this makes sense.

Cheers,

Post by Glyph Lefkowitz

On Dec 16, 2015, at 9:25 AM, Chris Norman
Hi all,
I'm writing a MUD server, and I want a way for transports to be
notified ofa shutdown before being disconnected, and the reactor
being stopped.
t.write('Shutting down.\r\n')
t.loseConnection()
reactor.stop()
This doesn't seem to notify the transports.
t.write('Shutting down.\r\n')
t.loseConnection()
pass
reactor.stop()
That just blocked and did nothing, presumably something do with my while loop.
Is there a stopWhenEmpty function on the somewhere? I did look over
the methods, and I couldn't find anything promising.
I'm just using the standard from twisted.internet import reactor
reactor, so no special cases here. In case it matters the transports
I'm using are twisted.protocols.basic.LineReceiver, and everything
else works with them.
Cheers in advance for the help.

This is definitely doable, but before I explain it would help to know
/why/ you want to do this.
The reason I ask is: servers crash; hardware fails. The falcon cannot
hear the falconer; things fall apart; the centre cannot hold.
When those servers /do/ crash (and they will), you don't get a clean
notification of disconnects. So if you're writing your application to
rely very heavily on the ability to do a clean shutdown and get
notifications of every disconnect at the time you expect
reactor.stop() to be running, you are probably designing your system
in a way that will be very fragile and prone to data loss. I know, I
have made this mistake more than once myself :).
So, before you continue: do you actually need to do this? Could you
just ignore the notification of the connection drop and exit
gracefully, perhaps properly cleaning up whatever state is left over
at next startup? If you really need this, understanding why you need
it would also help in determining which implementation technique to
suggest (there are a few).
-glyph
_______________________________________________
Twisted-Python mailing list
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Glyph Lefkowitz]

Hi,
It's a MUD server, so players type in commands and receive textual responses.
One of the admin commands is the ability to shutdown the server (or CTRL-C might be pressed on the console). I'd like this action to notify all connected transports that the server is going down for shutdown, so they're not rudely disconnected, then once the notifications have all gone through, then the server is free to shutdown.

Gotcha. So you don't need to necessarily wait for all the messages to be delivered if there are slow clients waiting around; you just want to send everyone a farewell message and if they haven't responded within a reasonable timeout, go ahead and shut down anyway.

If your MUD server is already a Service <https://twistedmatrix.com/documents/15.5.0/api/twisted.application.service.IService.html <https://twistedmatrix.com/documents/15.5.0/api/twisted.application.service.IService.html>> being launched by twistd, you just need to add a stopService <https://twistedmatrix.com/documents/15.5.0/api/twisted.application.service.IService.html#stopService <https://twistedmatrix.com/documents/15.5.0/api/twisted.application.service.IService.html#stopService>> method that returns a Deferred. When CTRL-C is hit (or anything else causes reactor.stop to be called), it will call this stopService method, and won't exit until a Deferred fires.

In your case, a simple deferLater <https://twistedmatrix.com/documents/15.5.0/api/twisted.internet.task.html#deferLater <https://twistedmatrix.com/documents/15.5.0/api/twisted.internet.task.html#deferLater>> will probably do the trick. You can also speed things up when there are no connected clients left by cancelling that Deferred to make it finish firing immediately.

Will that work for you?

I hope all this makes sense.

P.S.: For future reference, on this list the preferred style of reply is interleaved https://en.wikipedia.org/wiki/Posting_style#Interleaved_style <https://en.wikipedia.org/wiki/Posting_style#Interleaved_style> or bottom-posting: https://en.wikipedia.org/wiki/Posting_style#Bottom-posting <https://en.wikipedia.org/wiki/Posting_style#Bottom-posting>

-glyph

[Chris Norman]

Hello,

Post by Glyph Lefkowitz

On Dec 17, 2015, at 4:56 AM, Chris Norman
Hi,
It's a MUD server, so players type in commands and receive textual responses.
One of the admin commands is the ability to shutdown the server (or
CTRL-C might be pressed on the console). I'd like this action to
notify all connected transports that the server is going down for
shutdown, so they're not rudely disconnected, then once the
notifications have all gone through, then the server is free to shutdown.

Gotcha. So you don't need to necessarily wait for all the messages to
be delivered if there are slow clients waiting around; you just want
to send everyone a farewell message and if they haven't responded
within a reasonable timeout, go ahead and shut down anyway.
If your MUD server is already a Service
<https://twistedmatrix.com/documents/15.5.0/api/twisted.application.service.IService.html>
being launched by twistd, you just need to add a stopService
<https://twistedmatrix.com/documents/15.5.0/api/twisted.application.service.IService.html#stopService>
method that returns a Deferred. When CTRL-C is hit (or anything else
causes reactor.stop to be called), it will call this stopService
method, and won't exit until a Deferred fires.
In your case, a simple deferLater
<https://twistedmatrix.com/documents/15.5.0/api/twisted.internet.task.html#deferLater>
will probably do the trick. You can also speed things up when there
are no connected clients left by cancelling that Deferred to make it
finish firing immediately.
Will that work for you?

I hope all this makes sense.

It's not a service no... Should it be? I wasn't planning to use twistd,
mainly because I don't know how to, and running
python main.py
is working fine, accepting command line arguments - the works.

It could be converted though, if there is an advantage with services?

Also, I've read quite a lot about Deferreds. I thought initially they
were for multithreading your application, but I realise that's wrong, so
I don't understand what the point in them is?

This isn't to say there isn't one mind you, I think I'm just majorly
missing the point.

Post by Glyph Lefkowitz
P.S.: For future reference, on this list the preferred style of reply
is interleaved https://en.wikipedia.org
<https://en.wikipedia.org/wiki/Posting_style#Interleaved_style>

Sorry, I'll do that in the future.

Post by Glyph Lefkowitz
/wiki/Posting_style#Interleaved_style
<https://en.wikipedia.org/wiki/Posting_style#Interleaved_style> or
bottom-posting: https://en.wikipedia.org/wiki/Posting_style#Bottom-posting
-glyph
_______________________________________________
Twisted-Python mailing list
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Glyph Lefkowitz]

Post by Chris Norman
Hello,

Post by Glyph Lefkowitz

Hi,
It's a MUD server, so players type in commands and receive textual responses.
One of the admin commands is the ability to shutdown the server (or CTRL-C might be pressed on the console). I'd like this action to notify all connected transports that the server is going down for shutdown, so they're not rudely disconnected, then once the notifications have all gone through, then the server is free to shutdown.

Gotcha. So you don't need to necessarily wait for all the messages to be delivered if there are slow clients waiting around; you just want to send everyone a farewell message and if they haven't responded within a reasonable timeout, go ahead and shut down anyway.
If your MUD server is already a Service < <https://twistedmatrix.com/documents/15.5.0/api/twisted.application.service.IService.html>https://twistedmatrix.com/documents/15.5.0/api/twisted.application.service.IService.html <https://twistedmatrix.com/documents/15.5.0/api/twisted.application.service.IService.html>> being launched by twistd, you just need to add a stopService <https://twistedmatrix.com/documents/15.5.0/api/twisted.application.service.IService.html#stopService <https://twistedmatrix.com/documents/15.5.0/api/twisted.application.service.IService.html#stopService>> method that returns a Deferred. When CTRL-C is hit (or anything else causes reactor.stop to be called), it will call this stopService method, and won't exit until a Deferred fires.
In your case, a simple deferLater < <https://twistedmatrix.com/documents/15.5.0/api/twisted.internet.task.html#deferLater>https://twistedmatrix.com/documents/15.5.0/api/twisted.internet.task.html#deferLater <https://twistedmatrix.com/documents/15.5.0/api/twisted.internet.task.html#deferLater>> will probably do the trick. You can also speed things up when there are no connected clients left by cancelling that Deferred to make it finish firing immediately.
Will that work for you?

I hope all this makes sense.

It's not a service no... Should it be? I wasn't planning to use twistd, mainly because I don't know how to, and running
python main.py
is working fine, accepting command line arguments - the works.
It could be converted though, if there is an advantage with services?

Services are just the standard way to organize starting up and shutting down with Twisted. And running under `twistdŽ takes care of some of the details of starting up and hosting your service, including initializing logging, managing the timing of setting the UID and GID, selecting the reactor and so on. But if you're happy with your own service setup, it's not necessary. The alternative is to use `addSystemEventTriggerŽ: https://twistedmatrix.com/documents/15.5.0/api/twisted.internet.base.ReactorBase.html#addSystemEventTrigger <https://twistedmatrix.com/documents/15.5.0/api/twisted.internet.base.ReactorBase.html#addSystemEventTrigger>

Post by Chris Norman
Also, I've read quite a lot about Deferreds. I thought initially they were for multithreading your application, but I realise that's wrong, so I don't understand what the point in them is?

You can't block (i.e. wait to return from your function until you have a result) in an async system, so Deferreds are a placeholder for a result that doesn't exist yet. krondo.com's Twisted introduction covers this in some detail, if the official Twisted docs didn't help you: http://krondo.com/?page_id=1327

Post by Chris Norman
This isn't to say there isn't one mind you, I think I'm just majorly missing the point.

If there weren't a point to having Deferreds then Twisted would be a very strange system indeed ;).

-glyph