[/ Copyright Oliver Kowalke, Nat Goodspeed 2015. Distributed under the Boost Software License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt ] [/ import path is relative to this .qbk file] [#integration] [section:integration Sharing a Thread with Another Main Loop] [section Overview] As always with cooperative concurrency, it is important not to let any one fiber monopolize the processor too long: that could ["starve] other ready fibers. This section discusses a couple of solutions. [endsect] [section Event-Driven Program] Consider a classic event-driven program, organized around a main loop that fetches and dispatches incoming I/O events. You are introducing __boost_fiber__ because certain asynchronous I/O sequences are logically sequential, and for those you want to write and maintain code that looks and acts sequential. You are launching fibers on the application[s] main thread because certain of their actions will affect its user interface, and the application[s] UI framework permits UI operations only on the main thread. Or perhaps those fibers need access to main-thread data, and it would be too expensive in runtime (or development time) to robustly defend every such data item with thread synchronization primitives. You must ensure that the application[s] main loop ['itself] doesn[t] monopolize the processor: that the fibers it launches will get the CPU cycles they need. The solution is the same as for any fiber that might claim the CPU for an extended time: introduce calls to [ns_function_link this_fiber..yield]. The most straightforward approach is to call `yield()` on every iteration of your existing main loop. In effect, this unifies the application[s] main loop with __boost_fiber__[s] internal main loop. `yield()` allows the fiber manager to run any fibers that have become ready since the previous iteration of the application[s] main loop. When these fibers have had a turn, control passes to the thread[s] main fiber, which returns from `yield()` and resumes the application[s] main loop. [endsect] [#embedded_main_loop] [section Embedded Main Loop] More challenging is when the application[s] main loop is embedded in some other library or framework. Such an application will typically, after performing all necessary setup, pass control to some form of `run()` function from which control does not return until application shutdown. A __boost_asio__ program might call [@http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/io_service/run.html `io_service::run()`] in this way. In general, the trick is to arrange to pass control to [ns_function_link this_fiber..yield] frequently. You could use an [@http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/high_resolution_timer.html Asio timer] for that purpose. You could instantiate the timer, arranging to call a handler function when the timer expires. The handler function could call `yield()`, then reset the timer and arrange to wake up again on its next expiration. Since, in this thought experiment, we always pass control to the fiber manager via `yield()`, the calling fiber is never blocked. Therefore there is always at least one ready fiber. Therefore the fiber manager never calls [member_link algorithm..suspend_until]. Using [@http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/io_service/post.html `io_service::post()`] instead of setting a timer for some nonzero interval would be unfriendly to other threads. When all I/O is pending and all fibers are blocked, the io_service and the fiber manager would simply spin the CPU, passing control back and forth to each other. Using a timer allows tuning the responsiveness of this thread relative to others. [endsect] [section Deeper Dive into __boost_asio__] By now the alert reader is thinking: but surely, with Asio in particular, we ought to be able to do much better than periodic polling pings! [/ @path link is relative to (eventual) doc/html/index.html, hence ../..] This turns out to be surprisingly tricky. We present a possible approach in [@../../examples/asio/round_robin.hpp `examples/asio/round_robin.hpp`]. [import ../examples/asio/round_robin.hpp] [import ../examples/asio/autoecho.cpp] One consequence of using __boost_asio__ is that you must always let Asio suspend the running thread. Since Asio is aware of pending I/O requests, it can arrange to suspend the thread in such a way that the OS will wake it on I/O completion. No one else has sufficient knowledge. So the fiber scheduler must depend on Asio for suspension and resumption. It requires Asio handler calls to wake it. One dismaying implication is that we cannot support multiple threads calling [@http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/io_service/run.html `io_service::run()`] on the same `io_service` instance. The reason is that Asio provides no way to constrain a particular handler to be called only on a specified thread. A fiber scheduler instance is locked to a particular thread: that instance cannot manage any other thread[s] fibers. Yet if we allow multiple threads to call `io_service::run()` on the same `io_service` instance, a fiber scheduler which needs to sleep can have no guarantee that it will reawaken in a timely manner. It can set an Asio timer, as described above [mdash] but that timer[s] handler may well execute on a different thread! Another implication is that since an Asio-aware fiber scheduler (not to mention [link callbacks_asio `boost::fibers::asio::yield`]) depends on handler calls from the `io_service`, it is the application[s] responsibility to ensure that [@http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/io_service/stop.html `io_service::stop()`] is not called until every fiber has terminated. It is easier to reason about the behavior of the presented `asio::round_robin` scheduler if we require that after initial setup, the thread[s] main fiber is the fiber that calls `io_service::run()`, so let[s] impose that requirement. Naturally, the first thing we must do on each thread using a custom fiber scheduler is call [function_link use_scheduling_algorithm]. However, since `asio::round_robin` requires an `io_service` instance, we must first declare that. [asio_rr_setup] `use_scheduling_algorithm()` instantiates `asio::round_robin`, which naturally calls its constructor: [asio_rr_ctor] `asio::round_robin` binds the passed `io_service` pointer and initializes a [@http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/steady_timer.html `boost::asio::steady_timer`]: [asio_rr_suspend_timer] Then it calls [@http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/add_service.html `boost::asio::add_service()`] with a nested `service` struct: [asio_rr_service_top] ... [asio_rr_service_bottom] The `service` struct has a couple of roles. Its foremost role is to manage a [^std::unique_ptr<[@http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/io_service__work.html `boost::asio::io_service::work`]>]. We want the `io_service` instance to continue its main loop even when there is no pending Asio I/O. But when [@http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/io_service__service/shutdown_service.html `boost::asio::io_service::service::shutdown_service()`] is called, we discard the `io_service::work` instance so the `io_service` can shut down properly. Its other purpose is to [@http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/io_service/post.html `post()`] a lambda (not yet shown). Let[s] walk further through the example program before coming back to explain that lambda. The `service` constructor returns to `asio::round_robin`[s] constructor, which returns to `use_scheduling_algorithm()`, which returns to the application code. Once it has called `use_scheduling_algorithm()`, the application may now launch some number of fibers: [asio_rr_launch_fibers] Since we don[t] specify a [class_link launch], these fibers are ready to run, but have not yet been entered. Having set everything up, the application calls [@http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/io_service/run.html `io_service::run()`]: [asio_rr_run] Now what? Because this `io_service` instance owns an `io_service::work` instance, `run()` does not immediately return. But [mdash] none of the fibers that will perform actual work has even been entered yet! Without that initial `post()` call in `service`[s] constructor, ['nothing] would happen. The application would hang right here. So, what should the `post()` handler execute? Simply [ns_function_link this_fiber..yield]? That would be a promising start. But we have no guarantee that any of the other fibers will initiate any Asio operations to keep the ball rolling. For all we know, every other fiber could reach a similar `boost::this_fiber::yield()` call first. Control would return to the `post()` handler, which would return to Asio, and... the application would hang. The `post()` handler could `post()` itself again. But as discussed in [link embedded_main_loop the previous section], once there are actual I/O operations in flight [mdash] once we reach a state in which no fiber is ready [mdash] that would cause the thread to spin. We could, of course, set an Asio timer [mdash] again as [link embedded_main_loop previously discussed]. But in this ["deeper dive,] we[,]re trying to do a little better. The key to doing better is that since we[,]re in a fiber, we can run an actual loop [mdash] not just a chain of callbacks. We can wait for ["something to happen] by calling [@http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/io_service/run_one.html `io_service::run_one()`] [mdash] or we can execute already-queued Asio handlers by calling [@http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/io_service/poll.html `io_service::poll()`]. Here[s] the body of the lambda passed to the `post()` call. [asio_rr_service_lambda] We want this loop to exit once the `io_service` instance has been [@http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/io_service/stopped.html `stopped()`]. As long as there are ready fibers, we interleave running ready Asio handlers with running ready fibers. If there are no ready fibers, we wait by calling `run_one()`. Once any Asio handler has been called [mdash] no matter which [mdash] `run_one()` returns. That handler may have transitioned some fiber to ready state, so we loop back to check again. (We won[t] describe `awakened()`, `pick_next()` or `has_ready_fibers()`, as these are just like [member_link round_robin..awakened], [member_link round_robin..pick_next] and [member_link round_robin..has_ready_fibers].) That leaves `suspend_until()` and `notify()`. Doubtless you have been asking yourself: why are we calling `io_service::run_one()` in the lambda loop? Why not call it in `suspend_until()`, whose very API was designed for just such a purpose? Under normal circumstances, when the fiber manager finds no ready fibers, it calls [member_link algorithm..suspend_until]. Why test `has_ready_fibers()` in the lambda loop? Why not leverage the normal mechanism? The answer is: it matters who[s] asking. Consider the lambda loop shown above. The only __boost_fiber__ APIs it engages are `has_ready_fibers()` and [ns_function_link this_fiber..yield]. `yield()` does not ['block] the calling fiber: the calling fiber does not become unready. It is immediately passed back to [member_link algorithm..awakened], to be resumed in its turn when all other ready fibers have had a chance to run. In other words: during a `yield()` call, ['there is always at least one ready fiber.] As long as this lambda loop is still running, the fiber manager does not call `suspend_until()` because it always has a fiber ready to run. However, the lambda loop ['itself] can detect the case when no ['other] fibers are ready to run: the running fiber is not ['ready] but ['running.] That said, `suspend_until()` and `notify()` are in fact called during orderly shutdown processing, so let[s] try a plausible implementation. [asio_rr_suspend_until] As you might expect, `suspend_until()` sets an [@http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/steady_timer.html `asio::steady_timer`] to [@http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/basic_waitable_timer/expires_at.html `expires_at()`] the passed [@http://en.cppreference.com/w/cpp/chrono/steady_clock `std::chrono::steady_clock::time_point`]. Usually. As indicated in comments, we avoid setting `suspend_timer_` multiple times to the ['same] `time_point` value since every `expires_at()` call cancels any previous [@http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/basic_waitable_timer/async_wait.html `async_wait()`] call. There is a chance that we could spin. Reaching `suspend_until()` means the fiber manager intends to yield the processor to Asio. Cancelling the previous `async_wait()` call would fire its handler, causing `run_one()` to return, potentially causing the fiber manager to call `suspend_until()` again with the same `time_point` value... Given that we suspend the thread by calling `io_service::run_one()`, what[s] important is that our `async_wait()` call will cause a handler to run, which will cause `run_one()` to return. It[s] not so important specifically what that handler does. [asio_rr_notify] Since an `expires_at()` call cancels any previous `async_wait()` call, we can make `notify()` simply call `steady_timer::expires_at()`. That should cause the `io_service` to call the `async_wait()` handler with `operation_aborted`. The comments in `notify()` explain why we call `expires_at()` rather than [@http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/basic_waitable_timer/cancel.html `cancel()`]. [/ @path link is relative to (eventual) doc/html/index.html, hence ../..] This `boost::fibers::asio::round_robin` implementation is used in [@../../examples/asio/autoecho.cpp `examples/asio/autoecho.cpp`]. It seems possible that you could put together a more elegant Fiber / Asio integration. But as noted at the outset: it[s] tricky. [endsect] [endsect]