[Twisted-Python] Twisted and Flask or Django

[Craig Rodrigues]

Hi,

Moshe Zadka recently gave a very good presentation on using Twisted as a
WSGI container for Django or Flask apps at the San Francisco Django Meetup:

https://rawgit.com/moshez/talks/master/twist-wsgi/twist-wsgi.html

The examples in this presentation should be cleaned up and maybe put on the
front page of the twistedmatrix web site. Many Python developers are using
frameworks like Django and Flask. That would be a good way to get new
developers interested in Twisted.

--
Craig

[Pantelis Theodosiou]

Thank you, this looks very interesting.

As I was planning to ask a relevant question, let me add it here:

Are there any similar resources for other web frameworks, for example
Pyramid? And when I say resources, I mean talks, presentations or just a
web page with example code.

I think it would be helpful to have a page in the Twisted site, with links
to such resources - and perhaps a page for each (Django, Flask, Pyramid,
etc) with sample code and specific issues/tips than have been encountered
when combining Twisted with other frameworks.

Pantelis Theodosiou

Post by Craig Rodrigues
Hi,
Moshe Zadka recently gave a very good presentation on using Twisted as a
https://rawgit.com/moshez/talks/master/twist-wsgi/twist-wsgi.html
The examples in this presentation should be cleaned up and maybe put on
the front page of the twistedmatrix web site. Many Python developers are
using frameworks like Django and Flask. That would be a good way to get
new developers interested in Twisted.
--
Craig
_______________________________________________
Twisted-Python mailing list
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Glyph Lefkowitz]

Post by Pantelis Theodosiou
Thank you, this looks very interesting.
Are there any similar resources for other web frameworks, for example Pyramid? And when I say resources, I mean talks, presentations or just a web page with example code.

"Hendrix" is a tool for running various frameworks under Twisted; I'm pretty sure Pyramid is one of them (although I don't have a direct link to pyramid-specific docs handy):

https://hendrix.readthedocs.io/en/latest/ <https://hendrix.readthedocs.io/en/latest/>

For the most part, general WSGI documentation should get you most of the way there with specific frameworks though. This used to be a lot worse but in recent years there's been a lot of convergence.

Post by Pantelis Theodosiou
I think it would be helpful to have a page in the Twisted site, with links to such resources - and perhaps a page for each (Django, Flask, Pyramid, etc) with sample code and specific issues/tips than have been encountered when combining Twisted with other frameworks.

The Twisted site is a wiki, and if you are interested in curating such a page, I'm happy to give you the relevant edit permissions.

[Hynek Schlawack]

Post by Glyph Lefkowitz
For the most part, general WSGI documentation should get you most of the way there with specific frameworks though. This used to be a lot worse but in recent years there's been a lot of convergence.

I have to disagree here BTW. When I started using Twisted as a WSGI container, I was vexed by its lack of docs (and I’m still confused about many details). There’s a few blog posts around but none of them reach the straight-forwardibility of e.g. gunicorn’s docs. I’ve ended up asking around on IRC, stealing from Donald/PyPI, and reading source code.

I honestly believe, we’re leaving money on the table here. Using Twisted as a WSGI container should be first class use case on the web page since it presents a nice entry drug into the ecosystem.

There’s also barely any information on *why* one would want to use Twisted as a WSGI container. And please keep in mind that to many, many (most likely: the majority) of Python users “pure Python” doesn’t mean “safe” but “slow.” C.f. all those “lightning fast pure C” frameworks popping up left and right, upvoted to nirvana on reddit and its likes. If we want to do marketing, we have to be a tad more user-focused. :)

***

What I’m saying is that this topic shouldn’t be relegated into some wiki (where information famously is going to die) but should be a prominent tab on the landing page (maybe with additional information for specific frameworks in the wiki).

Cheers,
—hs

[Glyph Lefkowitz]

Post by Hynek Schlawack

Post by Glyph Lefkowitz
For the most part, general WSGI documentation should get you most of the way there with specific frameworks though. This used to be a lot worse but in recent years there's been a lot of convergence.

I have to disagree here BTW. When I started using Twisted as a WSGI container, I was vexed by its lack of docs (and I’m still confused about many details). There’s a few blog posts around but none of them reach the straight-forwardibility of e.g. gunicorn’s docs. I’ve ended up asking around on IRC, stealing from Donald/PyPI, and reading source code.

All I was saying here is that frameworks have gotten better about being "just WSGI" and not requiring a lot of crazy customization on the framework side of things. But, you're absolutely right that we need better docs, many parts of the setup are not obvious!

Post by Hynek Schlawack
I honestly believe, we’re leaving money on the table here. Using Twisted as a WSGI container should be first class use case on the web page since it presents a nice entry drug into the ecosystem.

Also agreed here.

Post by Hynek Schlawack
There’s also barely any information on *why* one would want to use Twisted as a WSGI container. And please keep in mind that to many, many (most likely: the majority) of Python users “pure Python” doesn’t mean “safe” but “slow.” C.f. all those “lightning fast pure C” frameworks popping up left and right, upvoted to nirvana on reddit and its likes. If we want to do marketing, we have to be a tad more user-focused. :)

Yep. And things like Moshe's talk are wonderful resources to try to illuminate this unfortunately dark area, so I do appreciate people taking the time to do them. I do also think that being a bit more strategic about it might result in vastly greater adoption though.

Post by Hynek Schlawack
***
What I’m saying is that this topic shouldn’t be relegated into some wiki (where information famously is going to die) but should be a prominent tab on the landing page (maybe with additional information for specific frameworks in the wiki).

The front page is also the wiki :-). I'm happy to have it there, I just think that it should perhaps be workshopped a little bit on a draft page first.

Post by Hynek Schlawack
Cheers,
—hs
_______________________________________________
Twisted-Python mailing list
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Hynek Schlawack]

Post by Glyph Lefkowitz

Post by Hynek Schlawack
I have to disagree here BTW. When I started using Twisted as a WSGI container, I was vexed by its lack of docs (and I’m still confused about many details). There’s a few blog posts around but none of them reach the straight-forwardibility of e.g. gunicorn’s docs. I’ve ended up asking around on IRC, stealing from Donald/PyPI, and reading source code.

All I was saying here is that frameworks have gotten better about being "just WSGI" and not requiring a lot of crazy customization on the framework side of things. But, you're absolutely right that we need better docs, many parts of the setup are not obvious!

Yeah, I didn’t actually mean to disagree as much as use it as an opportunity to voice something that has been brewing in me for a while. :)

Post by Glyph Lefkowitz

Post by Hynek Schlawack
There’s also barely any information on *why* one would want to use Twisted as a WSGI container. And please keep in mind that to many, many (most likely: the majority) of Python users “pure Python” doesn’t mean “safe” but “slow.” C.f. all those “lightning fast pure C” frameworks popping up left and right, upvoted to nirvana on reddit and its likes. If we want to do marketing, we have to be a tad more user-focused. :)

Yep. And things like Moshe's talk are wonderful resources to try to illuminate this unfortunately dark area, so I do appreciate people taking the time to do them. I do also think that being a bit more strategic about it might result in vastly greater adoption though.

Yeah Moshe’s evangelism made me interested in the topic but it’s hard to find anything more than his material. Particularly a good reference guide. As you wrote, the particular integration with web frameworks is trivial (but should be provided!)

Post by Glyph Lefkowitz

Post by Hynek Schlawack
What I’m saying is that this topic shouldn’t be relegated into some wiki (where information famously is going to die) but should be a prominent tab on the landing page (maybe with additional information for specific frameworks in the wiki).

The front page is also the wiki :-). I'm happy to have it there, I just think that it should perhaps be workshopped a little bit on a draft page first.

Heh absolutely. Just wanna stress that it should come right after “Web Server” in the “Code Examples” tabs. :)

***

Oh and I’d like to propose to officially merge the twisted-web mailing list into this one. I don’t think it makes any sense to separate those two anymore. It’s hard to do anything without web nowadays. :)

[Glyph Lefkowitz]

Oh and I’d like to propose to officially merge the twisted-web mailing list into this one. I don’t think it makes any sense to separate those two anymore. It’s hard to do anything without web nowadays. :)

I'm definitely inclined to agree about this. I don't think it's had enough independent activity to justify its continued existence. The perceived need for a separate list was a cultural feature of an earlier time :).

-glyph

[Phil Mayers]

Post by Hynek Schlawack
Oh and I’d like to propose to officially merge the twisted-web
mailing list into this one. I don’t think it makes any sense to

Seconded.

[Tom Uijldert]

-----Original Message-----
From: Twisted-Python [mailto:twisted-python-
Sent: 19 April 2017 12:34
Subject: Re: [Twisted-Python] Twisted and Flask or Django

Post by Hynek Schlawack
Oh and I’d like to propose to officially merge the twisted-web mailing
list into this one. I don’t think it makes any sense to

Seconded.

[Tom] hear hear...

[Craig Rodrigues]

Post by Hynek Schlawack
I have to disagree here BTW. When I started using Twisted as a WSGI
container, I was vexed by its lack of docs (and I’m still confused about
many details). There’s a few blog posts around but none of them reach the
straight-forwardibility of e.g. gunicorn’s docs. I’ve ended up asking
around on IRC, stealing from Donald/PyPI, and reading source code.
I honestly believe, we’re leaving money on the table here. Using Twisted
as a WSGI container should be first class use case on the web page since it
presents a nice entry drug into the ecosystem.

If someone was not dedicated to Twisted, why would they even bother going
through all the steps that you did to
figure out how to use Twisted as a WSGI container? I'm amazed that even
you did not give up. :)

Do you have any idea how Twisted fares in performance benchmarks of WSGI
containers?
Moshe's presentation didn't cover this, and I forgot to ask him at the
presentation.
I also couldn't find any WSGI benchmarks which included Twisted.

--
Craig

[Glyph Lefkowitz]

I have to disagree here BTW. When I started using Twisted as a WSGI container, I was vexed by its lack of docs (and I’m still confused about many details). There’s a few blog posts around but none of them reach the straight-forwardibility of e.g. gunicorn’s docs. I’ve ended up asking around on IRC, stealing from Donald/PyPI, and reading source code.
I honestly believe, we’re leaving money on the table here. Using Twisted as a WSGI container should be first class use case on the web page since it presents a nice entry drug into the ecosystem.
If someone was not dedicated to Twisted, why would they even bother going through all the steps that you did to
figure out how to use Twisted as a WSGI container? I'm amazed that even you did not give up. :)
Do you have any idea how Twisted fares in performance benchmarks of WSGI containers?
Moshe's presentation didn't cover this, and I forgot to ask him at the presentation.
I also couldn't find any WSGI benchmarks which included Twisted.

Do be honest, it probably doesn't do all that well. Making the container multi-process (as Moshe notes) would be a good first step to getting something worth benchmarking.

But, I should note, the obsession with benchmarks in WSGI containers is a symptom of a generally industry-wide lack of understanding of good performance analysis. The fastest WSGI container is probably 2% of your application's overhead; the slowest possible one, maybe 3%. The differences look huge when you're comparing servers against each other, but throw a real application and a database and some load balancer latency in there and the app server container utterly disappears. Even multi-process support isn't really about optimizing for performance (although certainly leveraging multiple cores helps), it's about being able to safely crash a WSGI container process without bringing down the whole server.

-glyph

[Glyph Lefkowitz]

Post by Glyph Lefkowitz
Do be honest

*facepalm*.

"To". To be honest.

[Pantelis Theodosiou]

Post by Pantelis Theodosiou
Are there any similar resources for other web frameworks, for example
Pyramid? And when I say resources, I mean talks, presentations or just a
web page with example code.
"Hendrix" is a tool for running various frameworks under Twisted; I'm
pretty sure Pyramid is one of them (although I don't have a direct link to
https://hendrix.readthedocs.io/en/latest/
For the most part, general WSGI documentation should get you most of the
way there with specific frameworks though. This used to be a lot worse but
in recent years there's been a lot of convergence.
I think it would be helpful to have a page in the Twisted site, with links
to such resources - and perhaps a page for each (Django, Flask, Pyramid,
etc) with sample code and specific issues/tips than have been encountered
when combining Twisted with other frameworks.
The Twisted site is a wiki, and if you are interested in curating such a
page, I'm happy to give you the relevant edit permissions.

Great, I'll be happy to do that.

[Glyph Lefkowitz]

Post by Pantelis Theodosiou
Great, I'll be happy to do that.

I think if you log in you should be able to create and edit most pages, although the front page will be locked. Let me know if that is not the case.

-g