We considered an I/O operation such as reading a file, and noted that there was a long delay between when the request is issued and when the response comes. If the operation is represented as a function call (such as input_line) that only returns when the response comes, then the program is unable to make use of the processor during that time.
We briefly mentioned threads, which allow a program to continue to make progress on multiple tasks simultaneously by asking the operating system to rapidly switch between them (creating the illusion that they happen at the same time).
We introduced asynchronous computation as an alternative model to threads. With asynchronous I/O, we allow the programmer to issue a request for input without waiting for a response. This lets the program make forward progress while waiting for the input.
We introduced some key concepts of the Async library
Rule 1: async functions are not allowed to block (i.e. wait for an external input)
functions that request input of type 'a return an empty box that will be filled (determined) when the input is ready: an 'a Deferred.t
the upon function allows programs to schedule functions to be run when a deferred becomes determined.
The scheduler runs when you call Scheduler.go, and also when any of the scheduled functions return. It (non-deterministically) chooses a function that is ready to run and runs it.
the bind function allows programs to easily chain asynchronous operations together.
>>= is a convenient notation for bind. It allows you to write asynchronous code that looks almost identical to its asynchronous counterpart
3110 Async docs. This is ``fake'' documentation: we have stripped down the official Async API documentation and added copious comments and descriptions. Not all of the functions, modules, and optional arguments are documented here, but there is a link to the full official documentation.
This lecture covers functions from the Async.Std module, the Deferred module, and the Reader module
Fall 2014 lecture notes, lectures 15 and 16
Consider the OCaml function input_line (in the Pervasives module). You might use it to read the first line of a file:
let file = open_in "foo.txt" in
let line = input_line file in
print_endline lineThe type of input_lines is
input_lines : in_channel -> stringIt reads a line from the given file and returns it.
But what /really/ happens? For full details, you should take architecture and OS courses, but we can give a caricature. The processor sends a read command over the bus to the hard drive. The hard drive physically moves a mechanical arm to the physical location of the data on the disk, and then it sends a message back to the CPU containing the data. The operating system then notices that your input is ready, and resumes the execution of your program.
This process can take on the order of 10ms (enough to sort an array of 5000 elements). During this time, the operating system schedules other programs to run on the CPU; your program can do nothing else because the call to input_line has not returned.
This is time that your program could be using to do other things. For example, if your program is a server, it might be servicing other clients during that time. If it is a game, it may be drawing graphics or performing a physics calculation. If it is a word processor or web browser, it might be updating the user interface in response to the user scrolling or pressing a button.
One approach to letting programs take advantage of this time is to use threads. Threads are an abstraction provided by the operating system that allow a program to seem to do multiple things at the same time. In reality, the processor rapidly switches between the threads of various running processes, providing the illusion of simultaneous execution.
If one of the threads of a program is blocked waiting for input, the operating system simply doesn't schedule it, but it does not block the other threads of the same program. This allows the program to continue to make forward progress.
In a sense, using threads to wait for I/O uses one illusion to cancel out the bad effects of another. The first illusion is that while I/O is happening, nothing else can (which is not true --- the OS just runs other programs while yours waits). The second is that if a program has multiple threads then it can do two things at the same time (which is also not true --- the OS just rapidly switches between multiple tasks).
While they make programming easier, these illusions come at a price. For programs that are doing lots of I/O (such as servers and desktop applications), creating lots of threads to have them sit around waiting for input can be prohibitively expensive. Because of this, an alternative approach is becoming popular for many applications.
In the next few lectures (and PS5) we will explore asynchronous programming.
In an asynchronous model, we forgo both of the above illusions. Instead, we allow the programmer to start an input operation without immediately waiting for the input to become available. The program can then do other useful things while waiting for the input.
In order to actually deal with the input when it does become available, we keep a list of callback functions that should be called sometime after their input arrives. A function called the scheduler, when run, selects one of the callbacks whose input has arrived, and calls it. When the callback returns, the scheduler the selects another callback, and so on.
There are two important rules to keep in mind:
Rule 1: Asynchronous functions are not allowed to block (i.e. wait for external events)
Rule 2: In the land of asynchrony, scheduler calls YOU. It only gets to run when your function returns. In particular, you cannot be interrupted. (and you block everyone else if you violate rule 1)
Jane Street's Async library enables asynchronous programming in OCaml. It provides useful abstractions for building asynchronous programs, an interface to various I/O devices (to asynchronously read files, wait for timeouts, communicate on the network, etc), and a scheduler.
The central type in Async is 'a Deferred.t. An 'a Deferred.t is a box that starts out empty (or undetermined) and can be filled with an 'a (at which point we say it is determined). Once determined, the value cannot be changed in the future.
Let's look at some code to see how this works. Let's start by rewriting the input_line code from above:
let run file =
open Async.Std
(** read from file *)
upon (Reader.read_line file)
(fun line -> print_endline line))Let's unpack this a bit. Reader.read_line has type
val Reader.read_line : Reader.t -> string Deferred.tWhen executed, it causes the read command to be sent to the disk, and then immediately returns an undetermined string Deferred.t.
upon has the type
val upon : 'a Deferred.t -> ('a -> unit) -> unitIt causes the function given by its second argument to be scheduled to run when the deferred given by its first argument becomes determined. In this case, the function (fun line -> print_endline line) is registered with the scheduler.
When run () is called by the scheduler, the read request will be issued, and the callback function will be scheduled. run will then return to the scheduler. The scheduler will choose another task to run (if one is available, it will wait if there is nothing left to do). At some point in the future, the read request will return a line of the file, and the corresponding Deferred.t will become determined. Sometime after that happens, the callback function will be called with the line passed in as an argument.
Here is another example. Consider the following code:
let run file =
upon (Reader.read_line file) (fun _ -> print_endline "line 1");
upon (after (Time.Span.sec 60)) (fun _ -> print_endline "line 2");
upon (after (Time.Span.sec 30)) (fun _ -> print_endline "line 3");
print_endline "line 4";
()There is a new function here: after has the type
val after : Core.Std.Time.Span.t -> unit Deferred.tIt returns a unit deferred that becomes determined after the given time span has elapsed.
What does run () do? The first thing it does is issue a read request to the given file, and associates a callback that prints "line 1" with it. It then starts a timer that will run for 60 seconds, and registers the function that prints "line 2" to be run when the timer determines its deferred. It calls after again to create a deferred that will become determined after 30 seconds, and registers the "line 3" callback to run when that deferred becomes determined. Finally it prints "line 4" and returns.
We are guaranteed that "line 4" will be printed first. The only way for the other three messages to be printed is if the scheduler invokes the corresponding callbacks, but once the scheduler starts running the "run" function, it will not run again until "run" returns.
Lines 1, 2, and 3 may be printed in any order. The scheduler only guarantees that the corresponding callbacks will be executed some time after the deferreds become determined; it gives no guarantees about when or in what order. In class, as we ran through the example, all three of our deferreds happened to be determined before we returned to the scheduler at all; at that point the scheduler is free to choose any of the three runnable callbacks to execute next.
To sequence together multiple operations, we can use the Deferred.bind function. Bind is similar to upon:
val upon : 'a Deferred.t -> ('a -> unit) -> unit val bind : 'a Deferred.t -> ('a -> 'b Deferred.t) -> 'b Deferred.t
[bind d f] runs f after d is determined; f should return a deferred; the deferred returned by bind becomes determined when the deferred returned by f is determined.
For example, suppose we wanted to write a function that begins the following sequence of operations and returns a deferred that becomes determined when they are all done:
we could write it as follows:
let first_line_of_file name =
bind (Reader.open_file name) (fun file ->
bind (Reader.read_line file) (fun line ->
return line
)
)return simply takes a normal value x of type 'a and creates a deferred that is immediately determined with the value x. We use it here because the second argument to bind needs to be a function that returns a deferred, not a normal value.
The infix operator (>>=) is shorthand for bind:
let (>>=) d f = bind d fit makes writing asynchronous functions very natural. You should read
d >>= fun x ->
e(which is parenthesized implicitly as (d) >>= (fun x -> e)) as
let x = d in
eor alternatively
> evaluate d (which may start a computation
> when d completes, run eFor example, we can rewrite the above example using this notation:
let first_line_of_file name =
Reader.open_file name >>= fun file ->
Reader.read_line file >>= fun line ->
return lineCompare this to the non-asynchronous version:
let first_line_of_file name =
let file = open_in name in
let line = input_line file in
line