asyncio
Asynchronous I/O, event loop, coroutines and tasks
Async IO?
Wikipedia: a form of input/output processing that permits other processing to continue before the transmission has finished.
Different solutions to the problem
Quick overview
Multiple processes
System resources.
Process communication is a problem.
Threads
Shared memory is an advantage
And a problem
GIL on CPython.
Green threads
Avoids synchronization problems. cooperative.
Managed in user-space.
No operating system optimization for native threads
Quite a hack on Python (gevent + monkey patching).
Asynchronous IO
Single thread, Single process, cooperative multitasking.
Reduces context switches and overhead
Need to make sure it's cooperative, CPU intensive code will block.
A trend started with the web?
Nope. GUIs been doing it for a long time.
- Main event loop
-
Event comes in (e.g: Button clicked)
- Handler called
- Back to the loop (step 1)
Good GUI developers know:
Don't block the main event loop (Looking
at you Swing).
Benefits for the Web
Handle many connections without an extra overhead.
Nginx, Lighttpd, NodeJS, Twisted, Tornado ...
What about Python?
Not new
asyncore & asynchat available since Python 1.5.2.
Uses select (or poll).
Virtually nobody used it.
Twisted
Tornado
gevent
asyncio
Uses epoll, kexec
Feedback from Twisted, Tornado, gevent, etc.
Inter operate, not replace them.
Defines an interface which can be implemented on top of other event loops.
Milestones
- Generators. 2.5 added the
send()method - 3.3:
yield from, generators can return values - PEP 3153 got people talking about it
- PEP 3156 tulip: reference implementation
- Turned into asyncio in 3.4
- Futures,
add_done_callback()for chaining. - 3.5:
async,await
Some terms
- Coroutines: a generator function, can receive values, decorated with
@coroutineorasync def - Future: promise of a result or an error
- Task: Future which runs a coroutine
From: http://www.slideshare.net/saghul/asyncio/35
See it for a in depth overview of the subject.
Some examples Python docs
https://docs.python.org/3/library/asyncio-task.htmlPython Async IO Resources
- 3rd party libraries
- Documentation
- Talks
- Tutorials
- Blog posts
Demo
- Agent collecting data from an api.
- Server listening for db changes, updates clients via web sockets.
aiohttp
HTTP client/server for asyncio
- Supports both HTTP Client and HTTP Server.
- Supports both Server WebSockets and Client WebSockets out-of-the-box.
- Web-server has Middlewares, Signals and pluggable routing.
RethinkDB
- Open-source, scalable document based database built from the ground up for the realtime web.
- Instead of polling for changes, the developer can tell RethinkDB to continuously push updated query results to applications in realtime.
- Supports asyncio's event loop since version 2.1 of the python driver.