Networking►Stanford CS144

lab0

Lab Checkpoint 0: networking warmup

pdf

(The lab documents are written closer to the level of detail that a software engineer will get from a boss or client. If you think something might not be fully specified, sometimes the truth is that it doesn’t matter—you could try one approach or the other and see what happens.)

1 Set up GNU/Linux on your computer

1
$ wget https://web.stanford.edu/class/cs144/vm_howto/setup_dev_env.sh

2 Networking by hand

2.1 Fetch a Web page

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
$ telnet cs144.keithw.org http
Trying 104.196.238.229...
Connected to cs144.keithw.org.
Escape character is '^]'.
GET /hello HTTP/1.1		# path to the file you are asking
Host: cs144.keithw.org  # server host name
Connection: close  #  This tells the server that you are finished  making requests, and it should close the connection as soon as it finishes replying     

HTTP/1.1 200 OK
Date: Thu, 16 Dec 2021 05:49:04 GMT
Server: Apache
Last-Modified: Thu, 13 Dec 2018 15:45:29 GMT
ETag: "e-57ce93446cb64"
Accept-Ranges: bytes
Content-Length: 14
Connection: close
Content-Type: text/plain

Hello, CS144!
Connection closed by foreign host.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
$ curl --verbose cs144.keithw.org/hello
> GET http://cs144.keithw.org/hello HTTP/1.1
> Host: cs144.keithw.org
> User-Agent: curl/7.58.0
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 OK
< Content-Length: 14
< Accept-Ranges: bytes
< Connection: keep-alive
< Content-Type: text/plain
< Date: Thu, 16 Dec 2021 06:04:03 GMT
< Etag: "e-57ce93446cb64"
< Keep-Alive: timeout=4
< Last-Modified: Thu, 13 Dec 2018 15:45:29 GMT
< Proxy-Connection: keep-alive
< Server: Apache
<
Hello, CS144!

if you want to do this with nc, beware that nc send the line each time you hit <Enter>, so there will be a 400 Bad Request

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
$ nc cs144.keithw.org http
GET /hello HTTP/1.1
HTTP/1.1 400 Bad Request
Date: Thu, 16 Dec 2021 06:09:33 GMT
Server: Apache
Content-Length: 226
Connection: close
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
</p>
</body></html>

use http in nc

just take a look at HTTP specification 5.2

1
2
3
4
5
6
7
8
9
10
11
12
$ printf 'GET /hello HTTP/1.1\r\nHost: cs144.keithw.org\r\nConnection: close\r\n\r\n' | nc cs144.keithw.org http
HTTP/1.1 200 OK
Date: Thu, 16 Dec 2021 06:08:02 GMT
Server: Apache
Last-Modified: Thu, 13 Dec 2018 15:45:29 GMT
ETag: "e-57ce93446cb64"
Accept-Ranges: bytes
Content-Length: 14
Connection: close
Content-Type: text/plain

Hello, CS144!

Assignment

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
$ printf 'GET /lab0/01 HTTP/1.1\r\nHost: cs144.keithw.org\r\nConnection: close\r\n\r\n' | nc cs144.keithw.org http
HTTP/1.1 200 OK
Date: Thu, 16 Dec 2021 06:14:24 GMT
Server: Apache
X-You-Said-Your-SunetID-Was: 01
X-Your-Code-Is: 966759
Content-length: 106
Vary: Accept-Encoding
Connection: close
Content-Type: text/plain

Hello! You told us that your SUNet ID was "01". Please see the HTTP headers (above) for your secret code.

$ curl cs144.keithw.org/lab0/01 --verbose
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 7890 (#0)
> GET http://cs144.keithw.org/lab0/01 HTTP/1.1
> Host: cs144.keithw.org
> User-Agent: curl/7.58.0
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 OK
< Connection: keep-alive
< Content-Type: text/plain
< Date: Thu, 16 Dec 2021 06:15:18 GMT
< Keep-Alive: timeout=4
< Proxy-Connection: keep-alive
< Server: Apache
< Vary: Accept-Encoding
< X-You-Said-Your-Sunetid-Was: 01
< X-Your-Code-Is: 109591
* no chunk, no close, no size. Assume close to signal end
<
Hello! You told us that your SUNet ID was "01". Please see the HTTP headers (above) for your secret code.

2.2 Send yourself an email

since we are not on stanford network and we don’t have a valid sunetid we cannot complish this part of the lab

2.3 Listening and connecting

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
$ netcat -v -l -p 9091
Listening on [0.0.0.0] (family 0, port 9091)
Connection from localhost 47546 received!
hello netcat
hello telnet
bye
bye
^C
$ telnet localhost 9091
Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
hello netcat
hello telnet
bye
bye
Connection closed by foreign host.

3 Writing a network program using an OS stream socket

In the next part of this warmup lab, you will write a short program that fetches a Web page over the Internet.

  • When two stream sockets are connected, any bytes written to one socket will eventually come out in the same order from the other socket on the other computer.

  • Although the network tries to deliver every datagram, in practice datagrams can be (1) lost, (2) delivered out of order, (3) delivered with the contents altered, or even (4) duplicated and delivered more than once. It’s normally the job of the operating systems on either end of the connection to turn “best-effort datagrams” (the abstraction the Internet provides) into “reliable byte streams” (the abstraction that applications usually want).

  • The two computers have to cooperate to make sure that each byte in the stream eventually gets delivered, in its proper place in line, to the stream socket on the other side. This was done by TCP

1
2
3
4
5
6
$ git clone https://github.com/cs144/sponge
$ cd sponge
$ mkdir build
$ cd build
$ cmake ..
$ make
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
$ cmake -DCMAKE_CXX_COMPILER=`which g++-8` -DCMAKE_C_COMPILER=`which gcc-8` ..
-- The C compiler identification is GNU 8.4.0
-- The CXX compiler identification is GNU 8.4.0
-- Check for working C compiler: /usr/bin/gcc-8
-- Check for working C compiler: /usr/bin/gcc-8 -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/g++-8
-- Check for working CXX compiler: /usr/bin/g++-8 -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Setting build type to 'Release'
--   NOTE: You can choose a build type by calling cmake with one of:
--     -DCMAKE_BUILD_TYPE=Release   -- full optimizations
--     -DCMAKE_BUILD_TYPE=Debug     -- better debugging experience in gdb
--     -DCMAKE_BUILD_TYPE=RelASan   -- full optimizations plus address and undefined-behavior sanitizers
--     -DCMAKE_BUILD_TYPE=DebugASan -- debug plus sanitizers
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- Configuring done
-- Generating done
-- Build files have been written to: /home/vagrant/CS144/lab0/3/sponge/build
$ make
[100%] Built target parser_dt

My solution:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
void get_URL(const string &host, const string &path) {
    // Your code here.
    // You will need to connect to the "http" service on
    // the computer whose name is in the "host" string,
    Address target = Address(host , "http");
    TCPSocket request ;
    request.connect(target);

    // then request the URL path given in the "path" string.

    request.write("GET " + path + " HTTP/1.1\r\n");
    request.write("Host: " + host + "\r\n");
    request.write("Connection: close\r\n\r\n");

    // Then you'll need to print out everything the server sends back,
    // (not just one call to read() -- everything) until you reach
    // the "eof" (end of file).

    string received_content ;
    while(!request.eof()){
        received_content += request.read();
    }
1
2
3
4
5
6
7
8
9
10
$ make check_webget
[100%] Testing webget...
Test project /home/vagrant/CS144/lab0/3/sponge/build
    Start 31: t_webget
1/1 Test #31: t_webget .........................   Passed    1.03 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   1.03 sec
[100%] Built target check_webget

4 An in-memory reliable byte stream

just a simple std::deque<char>

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
#ifndef SPONGE_LIBSPONGE_BYTE_STREAM_HH
#define SPONGE_LIBSPONGE_BYTE_STREAM_HH

#include <string>
#include <deque>
using std::deque ;

//! \brief An in-order byte stream.

//! Bytes are written on the "input" side and read from the "output"
//! side.  The byte stream is finite: the writer can end the input,
//! and then no more bytes can be written.
class ByteStream {
  private:
    // Your code here -- add private members as necessary.
    // Hint: This doesn't need to be a sophisticated data structure at
    // all, but if any of your tests are taking longer than a second,
    // that's a sign that you probably want to keep exploring
    // different approaches.
    const size_t maxlen ;
    deque<char> slot ;
    bool _end = false ;
    size_t write_cot ;
    size_t read_cot  ;
    bool _error = false ;  //!< Flag indicating that the stream suffered an error.

  public:
    //! Construct a stream with room for `capacity` bytes.
    ByteStream(const size_t capacity);

    //! \name "Input" interface for the writer
    //! Write a string of bytes into the stream. Write as many
    //! as will fit, and return how many were written.
    //! \returns the number of bytes accepted into the stream
    size_t write(const std::string &data);

    //! \returns the number of additional bytes that the stream has space for
    size_t remaining_capacity() const;

    //! Signal that the byte stream has reached its ending
    void end_input();

    //! Indicate that the stream suffered an error.
    void set_error() { _error = true; }

    //! Peek at next "len" bytes of the stream
    //! \returns a string
    std::string peek_output(const size_t len) const;

    //! Remove bytes from the buffer
    void pop_output(const size_t len);

    //! Read (i.e., copy and then pop) the next "len" bytes of the stream
    //! \returns a string
    std::string read(const size_t len);

    //! \returns `true` if the stream input has ended
    bool input_ended() const;

    //! \returns `true` if the stream has suffered an error
    bool error() const { return _error; }

    //! \returns the maximum amount that can currently be read from the stream
    size_t buffer_size() const;

    //! \returns `true` if the buffer is empty
    bool buffer_empty() const;

    //! \returns `true` if the output has reached the ending
    bool eof() const;

    //! Total number of bytes written
    size_t bytes_written() const;

    //! Total number of bytes popped
    size_t bytes_read() const;
};
#endif  // SPONGE_LIBSPONGE_BYTE_STREAM_HH
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65

#include "byte_stream.hh"
#include <iostream>

// Dummy implementation of a flow-controlled in-memory byte stream.

// For Lab 0, please replace with a real implementation that passes the
// automated checks run by `make check_lab0`.

// You will need to add private members to the class declaration in `byte_stream.hh`

using namespace std;

ByteStream::ByteStream(const size_t capacity) :  maxlen (capacity)  , slot(deque<char>()) , write_cot (0) , read_cot(0){
}

size_t ByteStream::write(const string &data) {
    auto asklen = data.length();
    auto end = remaining_capacity() >= asklen ? asklen : remaining_capacity() ;
    for (size_t i = 0 ; i < end ; i++ ){
        slot.push_back(data.at(i));
    }
    write_cot += end ;
    return end ;
}

//! \param[in] len bytes will be copied from the output side of the buffer
string ByteStream::peek_output(const size_t len) const {
    auto it = slot.cbegin();
    return string(it , it + len) ;
}

//! \param[in] len bytes will be removed from the output side of the buffer
void ByteStream::pop_output(const size_t len) {
    for (size_t i = 0 ; i < len ; i++){
    slot.pop_front();
    }
    read_cot += len ;
}

//! Read (i.e., copy and then pop) the next "len" bytes of the stream
//! \param[in] len bytes will be popped and returned
//! \returns a string
std::string ByteStream::read(const size_t len) {
    string res = peek_output(len);
    pop_output(len);
    return res;
}

void ByteStream::end_input() { _end = true ; }

bool ByteStream::input_ended() const { return _end ; }

size_t ByteStream::buffer_size() const { return slot.size(); }

bool ByteStream::buffer_empty() const { return slot.empty() ; }

bool ByteStream::eof() const { return buffer_empty() && input_ended() ; }

size_t ByteStream::bytes_written() const { return write_cot ;  }

size_t ByteStream::bytes_read() const { return read_cot ; }

size_t ByteStream::remaining_capacity() const { return maxlen - slot.size(); }

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
$ make
$ make check_lab0
[100%] Testing Lab 0...
Test project /home/vagrant/CS144/lab0/3/sponge/build
    Start 26: t_byte_stream_construction
1/9 Test #26: t_byte_stream_construction .......   Passed    0.00 sec
    Start 27: t_byte_stream_one_write
2/9 Test #27: t_byte_stream_one_write ..........   Passed    0.00 sec
    Start 28: t_byte_stream_two_writes
3/9 Test #28: t_byte_stream_two_writes .........   Passed    0.00 sec
    Start 29: t_byte_stream_capacity
4/9 Test #29: t_byte_stream_capacity ...........   Passed    0.33 sec
    Start 30: t_byte_stream_many_writes
5/9 Test #30: t_byte_stream_many_writes ........   Passed    0.00 sec
    Start 31: t_webget
6/9 Test #31: t_webget .........................   Passed    1.06 sec
    Start 53: t_address_dt
7/9 Test #53: t_address_dt .....................   Passed    0.01 sec
    Start 54: t_parser_dt
8/9 Test #54: t_parser_dt ......................   Passed    0.00 sec
    Start 55: t_socket_dt
9/9 Test #55: t_socket_dt ......................   Passed    0.00 sec

100% tests passed, 0 tests failed out of 9

Total Test time (real) =   1.42 sec
[100%] Built target check_lab0