Clever Geek Handbook

Command Line Tool Development: Comparing Go and Rust

This article explores my experiment in writing a small command line tool using two languages ​​that I don't have much programming experience in. It's about Go and Rust. If you can't wait to see the code and independently compare one version of my program with another, then here is the repository of the Go version of the project, and here is the repository of its version written in Rust.











Project overview



I have a home project called Hashtrack. This is a small site, full stack application that I wrote for a technical interview. It's very easy to work with it:



  1. The user is authenticated (given that he has already created an account for himself).
  2. He introduces hashtags that he wants to watch appear on Twitter.
  3. He waits for the found tweets with the specified hashtag to appear on the screen.


You can try Hashtrack here .



After completing the interview, I continued to work on the project out of sports interest, and I noticed that it can be a great platform on which I can test my knowledge and skills in the field of developing command line tools. I already had a server, so I just had to choose a language in which I would implement a small set of capabilities within the API of my project.



Command line tool capabilities



Here is a description of the main features, in particular - the commands that I wanted to implement in my command line tool.



  • hashtrack login - logging into the system, that is - creating a session token and saving it in the local file system, in the configuration file.
  • hashtrack logout — , — , .
  • hashtrack track <hashtag> [...] — .
  • hashtrack untrack <hashtag> [...] — .
  • hashtrack tracks — , .
  • hashtrack list — 50 .
  • hashtrack watch — .
  • hashtrack status — , .
  • --endpoint, .
  • --config, .
  • endpoint.


Here are some important things to keep in mind about my tool before starting work on it:



  • It should use the project API that uses GraphQL, HTTP and WebSocket.
  • It must use the file system to store the configuration file.
  • It should be able to parse positional arguments and command line flags.


Why did I decide to use Go and Rust?



There are many languages ​​in which you can write command line tools.



In this case, I wanted to choose a language that I had no experience with, or a language with which I had very little experience. In addition, I wanted to find something that would easily compile to machine code, as this is an added plus for a command line tool.



The first language, which is obvious to me, came to my mind Go. This is probably because a lot of the command line tools I use are written in Go. But I also had a little experience in Rust programming, and it seemed to me that this language would also be well suited for my project.



Thinking about Go and Rust, I thought that you can choose both languages. Since my main goal was self-study, such a move would give me an excellent opportunity to implement the project twice and independently figure out the advantages and disadvantages of each of the languages.



Here I would like to mention the languages Crystal and Nim . They look promising. I am looking forward to the opportunity to test them in my next project.



Local environment



Before using a new set of tools, I am always interested in its usability. Namely, whether I have to use some kind of package manager to globally install programs on the system. Or, which seems to me a much more convenient solution, whether it will be possible to install everything based on the user account. We are talking about version managers, they simplify our life, focusing on installing programs on users, and not on the system as a whole. In the Node.js environment, NVM does this very well .



When working with Go, you can use the GVM for the same purpose . This project is responsible for local software installation and version control. Installing it is very simple: 



gvm install go1.14 -B
gvm use go1.14


When preparing a development environment in Go, you need to be aware of the existence of two environment variables - GOROOTand GOPATH. You can read more about them here .



The first problem I faced using Go was as follows. When I tried to understand how the module resolution system works and how it is applied GOPATH, it was quite difficult for me to set up a project structure with a functional local development environment.



I ended up just using the project directory GOPATH=$(pwd). The main plus of this was that I had a system for working with dependencies at my disposal, limited by the framework of a separate project, something like node_modules. This system has performed well.



After I finished working on my tool, I found that there was a virtualgo project that would help me solve my problems with GOPATH.



Rust has an official rustup installer that installs the toolkit needed to use Rust. Rust can be installed with literally one command. In addition, when used rustup, we have access to additional components such as the rls server and the rustfmt code formatter . Many projects require nightly builds of the Rust toolbox. Thanks to the application rustup, I had no problem switching between versions.



Editor support



I am using VS Code and was able to find extensions that target Go and Rust. Both languages ​​are perfectly supported in the editor.



To debug Rust code, following this tutorial, I needed to install the CodeLLDB extension .



Package management



The Go ecosystem doesn't have a package manager or even an official registry. Here the module resolution system is based on importing modules from external URLs.



Rust uses the Cargo package manager to manage dependencies, which downloads packages from crates.io from the official Rust package registry. In packages of ecosystem Crates can be documentation posted on docs.rs .



Libraries



My first goal in exploring new languages ​​was to figure out how difficult it would be to implement simple HTTP communication with a GraphQL server using requests and mutations.



Speaking of Go, I managed to find several libraries like machinebox / graphql and shurcooL / graphql . The second one uses structures for marshaling and unmarshaling data. Therefore, I chose her.



I forked shurcooL / graphql as I needed to customize the header on the client Authorization. Changes are submitted by this PR.



Here is an example of invoking a GraphQL mutation written in Go: 



type creationMutation struct {
    CreateSession struct {
        Token graphql.String
    } `graphql:"createSession(email: $email, password: $password)"`
}

type CreationPayload struct {
    Email    string
    Password string
}

func Create(client *graphql.Client, payload CreationPayload) (string, error) {
    var mutation creationMutation
    variables := map[string]interface{}{
        "email":    graphql.String(payload.Email),
        "password": graphql.String(payload.Password),
    }
    err := client.Mutate(context.Background(), &mutation, variables)

    return string(mutation.CreateSession.Token), err
}


When using Rust, I needed to use two libraries to execute GraphQL queries. The point here is that the library is graphql_clientprotocol independent, it is aimed at generating code for serializing and deserializing data. Therefore, I needed a second library ( reqwest), with which I organized the work with HTTP requests. 



#[derive(GraphQLQuery)]
#[graphql(
    schema_path = "graphql/schema.graphql",
    query_path = "graphql/createSession.graphql"
)]
struct CreateSession;

pub struct Session {
    pub token: String,
}

pub type Creation = create_session::Variables;

pub async fn create(context: &Context, creation: Creation) -> Result<Session, api::Error> {
    let res = api::build_base_request(context)
        .json(&CreateSession::build_query(creation))
        .send()
        .await?
        .json::<Response<create_session::ResponseData>>()
        .await?;
    match res.data {
        Some(data) => Ok(Session {
            token: data.create_session.token,
        }),
        _ => Err(api::Error(api::get_error_message(res).to_string())),
    }
}


None of the Go and Rust libraries supported GraphQL over the WebSocket protocol.



In fact, the library graphql_clientsupports subscriptions, but since it is protocol independent, I had to implement the WebSocket interaction mechanisms with GraphQL myself.



To use WebSocket in the Go version of the application, the library had to be modified. Since I already used a fork of the library, I didn't want to do this. Instead, I used a simplified way of “watching” new tweets. Namely, to receive tweets, I sent API requests every 5 seconds. I’m not proud to do just that .



When writing programs in Go, you can use the keywordgoto run lightweight streams called goroutines. Rust uses operating system threads, this is done by calling Thread::spawn. Channels are used to transfer data between streams and there and there.



Error processing



Go treats errors the same way as any other value. The usual way to handle errors in Go is to check for errors: 



func (config *Config) Save() error {
    contents, err := json.MarshalIndent(config, "", "    ")
    if err != nil {
        return err
    }

    err = ioutil.WriteFile(config.path, contents, 0o644)
    if err != nil {
        return err
    }

    return nil
}


Rust has an enumeration Result<T, E>that includes values ​​that represent success and failure. This, respectively, Ok(T)and Err(E). There is another enumeration here Option<T>that includes the values Some(T)and None. If you are familiar with Haskell, then you can recognize the monads Eitherand in these meanings Maybe.



There is also "syntactic sugar" related to error propagation (operator ?), which resolves the value of the structure Resulteither Optionand automatically returns Err(...)or Noneif something goes wrong. 



pub fn save(&mut self) -> io::Result<()> {
    let json = serde_json::to_string(&self.contents)?;
    let mut file = File::create(&self.path)?;
    file.write_all(json.as_bytes())
}


This code is equivalent to the following code: 



pub fn save(&mut self) -> io::Result<()> {
    let json = match serde_json::to_string(&self.contents) {
        Ok(json) => json,
        Err(e) => return Err(e.into())
    };
    let mut file = match File::create(&self.path) {
        Ok(file) => file,
        Err(e) => return Err(e.into())
    };
    file.write_all(json.as_bytes())
}


So, Rust has the following:



  • Monadic structure ( Optionand Result).
  • Operator support ?.
  • A trait Fromused to automatically convert errors as they propagate.


The combination of the above three features gives us an error handling system that I would call the best I've seen. It is simple and streamlined, and the code written using it is easy to maintain.



Compilation time



Go is a language that was created with the idea that the code written in it would compile as quickly as possible. Let's examine this question: 



> time go get hashtrack #  
go get hashtrack  1,39s user 0,41s system 43% cpu 4,122 total

> time go build -o hashtrack hashtrack #  
go build -o hashtrack hashtrack  0,80s user 0,12s system 152% cpu 0,603 total

> time go build -o hashtrack hashtrack #  
go build -o hashtrack hashtrack  0,19s user 0,07s system 400% cpu 0,065 total

> time go build -o hashtrack hashtrack #      
go build -o hashtrack hashtrack  0,94s user 0,13s system 169% cpu 0,629 total


Impressive. Now let's see what Rust will show us: 



> time cargo build
   Compiling libc v0.2.67
   Compiling cfg-if v0.1.10
   Compiling autocfg v1.0.0
   ...
   ...
   ...
   Compiling hashtrack v0.1.0 (/home/paulo/code/cuchi/hashtrack/cli-rust)
    Finished dev [unoptimized + debuginfo] target(s) in 1m 44s
cargo build  363,80s user 17,05s system 365% cpu 1:44,09 total


All dependencies are compiled here, which is 214 modules. When you restart compilation, everything is already prepared, so this task is performed almost instantly: 



> time cargo build #  
    Finished dev [unoptimized + debuginfo] target(s) in 0.08s
cargo build  0,07s user 0,03s system 104% cpu 0,094 total

> time cargo build #      
   Compiling hashtrack v0.1.0 (/home/paulo/code/cuchi/hashtrack/cli-rust)
    Finished dev [unoptimized + debuginfo] target(s) in 3.15s
cargo build  3,01s user 0,52s system 111% cpu 3,162 total


As you can see, Rust uses an incremental compilation model. Partial recompilation of the dependency tree is performed, starting with the modified module and ending with the modules that depend on it.



The release build of the project takes more time, which is quite expected, since the compiler optimizes the code in this case: 



> time cargo build --release
   Compiling libc v0.2.67
   Compiling cfg-if v0.1.10
   Compiling autocfg v1.0.0
   ...
   ...
   ...
   Compiling hashtrack v0.1.0 (/home/paulo/code/cuchi/hashtrack/cli-rust)
    Finished release [optimized] target(s) in 2m 42s
cargo build --release  1067,72s user 16,95s system 667% cpu 2:42,45 total


Continuous integration



The features of compiling projects written in Go and Rust, which we identified above, appear, which is quite expected, in the continuous integration system.





Go project processing





Processing a Rust project



Memory consumption



To analyze the memory consumption of different versions of my command line tool, I used the following command: 



/usr/bin/time -v ./hashtrack list


The command time -vdisplays a lot of interesting information, but I was interested in the process indicator Maximum resident set size, which is the peak amount of physical memory allocated to a program during its execution.



Here's the code I used to collect memory consumption data for different versions of the program: 



for n in {1..5}; do
    /usr/bin/time -v ./hashtrack list > /dev/null 2>> time.log
done
grep 'Maximum resident set size' time.log


Here are the results for the Go version: 



Maximum resident set size (kbytes): 13632
Maximum resident set size (kbytes): 14016
Maximum resident set size (kbytes): 14244
Maximum resident set size (kbytes): 13648
Maximum resident set size (kbytes): 14500


Here is the memory consumption of the Rust version of the program: 



Maximum resident set size (kbytes): 9840
Maximum resident set size (kbytes): 10068
Maximum resident set size (kbytes): 9972
Maximum resident set size (kbytes): 10032
Maximum resident set size (kbytes): 10072


This memory is allocated during the following tasks:



  • Interpretation of system arguments.
  • Loading and parsing the configuration file from the file system.
  • Accessing GraphQL over HTTP using TLS.
  • Parsing the JSON response.
  • Writing formatted data to stdout.


Go and Rust have different ways of managing memory.



Go has a garbage collector that is used to detect unused memory and reclaim it. As a result, the programmer is not distracted by these tasks. Since garbage collector is based on heuristic algorithms, using it always means making compromises. Typically between performance and the amount of memory used by the application.



Rust's memory management model has concepts like ownership, borrowing, lifetime. This not only contributes to safe memory handling, but also ensures complete control over memory allocated on the heap without requiring manual memory management or garbage collection.



For comparison, let's look at other programs that solve a problem similar to mine.



Command Maximum resident set size (kbytes)
heroku apps 56436
gh pr list 26456
git ls-remote (with SSH access) 6448
git ls-remote (with HTTP access) 23488


Reasons why I would choose Go



I would choose Go for some project for the following reasons:



  • If I needed a language that would be easy for my team members to learn.
  • If I wanted to write simple code at the expense of less flexibility of the language.
  • If I was developing software only for Linux, or if Linux was the operating system of most interest to me.
  • If the compilation time of projects was important.
  • If I needed mature mechanisms for asynchronous code execution.


Reasons why I would choose Rust



Here are the reasons that might lead me to choose Rust for a project:



  • If I needed an advanced error handling system.
  • If I wanted to write in a multi-paradigm language that allows me to write more expressive code than I could create with other languages.
  • If my project had very high security requirements.
  • If high performance was vital to the project.
  • If the project was aimed at many operating systems and I would like to have a truly multi-platform codebase.


General remarks



Go and Rust have some quirks that still haunt me. These are the following:



  • Go is so focused on simplicity that sometimes this pursuit has the opposite effect (for example, as in the cases with GOROOTand GOPATH).
  • I still don't really understand the concept of "lifetime" in Rust. Even attempts to work with the corresponding mechanisms of language throw me off balance.


Yes, I want to note that in newer versions of Go, working with GOPATHno longer causes problems, so I should transfer my project to a newer version of Go.



I can say that both Go and Rust are languages ​​that were very interesting to learn. I find them to be great additions to the capabilities of the C / C ++ programming world. They allow you to create applications of a wide variety of purposes. For example, web services and even, thanks to WebAssembly, client-side web applications .



Outcome



Go and Rust are great tools well suited for developing command line tools. But, of course, their creators were guided by different priorities. One language aims to make software development simple and accessible, so that code written in that language is maintainable. The other language's priorities are rationality, safety, and performance.



If you want to read more about Go vs Rust comparison, take a look at this article. Among other things, it raises an issue concerning serious problems with multi-platform compatibility of programs.



What language would you use to develop a command line tool?





More articles:


All Articles