An overview of SwiftSyntaxLuciano AlmeidaBlockedUnblockFollowFollowingApr 14SwiftSyntax is a library that provides a Swift abstraction on top of the libSyntax, providing a set of APIs that makes possible to do things like visiting, rewriting and retrieve information from the syntactic structure of a swift source.
So, in today’s article, we are going to play around and explore a little bit the SwiftSyntax library to understand more about how it works and how we can use to create things that can help us solve some problems.
So, let’s dive in …The CompilerBefore we dive into the SwiftSyntax, we need to understand at least in a high level some things about the compiler flow.
https://www.
draw.
io/The swift compiler takes the Swift source code, handles it to a hand-coded Lexer that tokenizes and transforms it into an Abstract Syntax Tree(AST), after that there is the Semantic Analysis (Sema) where the compiler takes the AST generated by the parser and makes a type-checked AST and check for semantic issues on that.
Then the Swift Intermediate Language Generation (SILGen) phase transforms the AST generated by the semantic analysis into what they call raw SIL, after some optimizations on the SIL such as generic specialization and ARC optimizations it passes this optimized SIL to the IRGen to generate the Intermediate Representation(IR) to be passed to LLVM for it to continue the job and generate the object files (.
o) that later will be glued together by the linker and generate the final binary for any given platforms.
That’s is a brief high-level overview of the compiler pipeline.
Here we are going to focus more on the AST because on SwiftSyntax we work basically on a representation of it in the form of Syntax nodes.
So, to understand the terminology in this post, we just need to know that an AST is the representation of a source code syntax in a form of tree structure.
libSyntaxWe can say that SwiftSyntax is a swift package that provides a set of swift bindings on top of libSyntax which is where the implementation, structure and logic to deal with the swift source it at.
On the libSyntax read me:This library implements data structures and algorithms for dealing with Swift syntax, striving to be safe, correct, and intuitive to use.
The library emphasizes immutable, thread-safe data structures, full-fidelity representation of source, and facilities for structured editing.
In other words, it will provide the basic blocks necessary for us to perform safe and reliable analysis or editing on a swift source syntactic structure with a nice and easy API.
SwiftSyntax APIWe’ll not be going to dive deep into much detail here on how the source data is represented internally in the libSyntax, so we’ll focus more on the high-level swift API provided by SwiftSyntax that we use as a client.
There are in detail documentation on it in the Internals section from the libSyntax readme file.
And also, there’s a section in the Harlan Haskins talk on try!.Swift NYC 2017[3] about how libSyntax represents the syntax tree internally.
But a brief summary, the libSyntax divides the representation of the AST in:Syntax: Or Syntax Nodes are the representation provided for the public API.
RawSyntax: Is the internal raw immutable backing store for all Syntax.
Store data like token kind and also represents the subtree structure.
RawTokenSyntax: special cases of RawSyntax and represent all terminals in the grammar.
Trivia: represents all the syntax parts that have no semantic meaning to the source, e.
g.
whitespaces, newlines and comments.
SyntaxData: It wraps RawSyntax nodes with some additional information: a pointer to a parent, the position in which the node occurs in its parent, and cached children.
High-level APIHere we are going to see a few high-level APIs that we can use on SwiftSyntax and some sample codes of how to use it.
SyntaxFactoryProvide a nice and simple API for creating any SyntaxNode in a single line way.
So instead of having many constructors on each Syntax class, we use the factory class.
With APIsEvery SyntaxNode have with methods that allow us to create a node from another with only the with part modification.
So, because the Syntax nodes have the concept of immutability it creates a new node with the same data than the node it was called on, but with the with part replaced.
SyntaxVisitorsWith the SyntaxVisitor, we can walk through the syntax tree.
It is useful when we want to extract some information to perform analysis on the source.
The return is a continuation kind indicating whether to continue and visit the children (SyntaxVisitorContinueKind.
visitChildren) nodes on the Syntax tree or skip it (SyntaxVisitorContinueKind.
skipChildren).
SyntaxRewritersThe SyntaxRewriter allows us to modify the structure of the tree by just overriding the visit(some syntax …) method and returning the new node based on a rule.
Note: All the nodes are immutable, so we don’t modify a node, but instead, we create another (using a with API) and return it to replace the current one.
In the example above we replace all the string literals on our code for a ????.
There is a lot more on the public API, but I think those are the main basic APIs that allows us to get started with SwiftSyntax.
ConclusionAlthough there is a note in the SwiftSyntax readme saying:Note: SwiftSyntax is still in development, and the API is not guaranteed to be stable.
It’s subject to change without warning.
we can already do cool things with it, and there is already a bunch of tools that people are using that are powered by SwiftSyntax.
There are tools for code formatting, for detect unused code and even the Swift Stress Tester is build on top of it, and there is a lot more.
To wrap up this article, a shot out to the teams and people involved on the libSyntax and SwiftSyntax, although is still a work in progress, is amazing the possibilities of what we can do with it, so congrats to the teams and people involved.
In the next post we are going to explore an example of how to create a very simple tool using SwiftSyntax and SPM.
That’s all for this article o/If you got some comment or question, please let me know.
Your feedback is really important so we can improve this and the future posts and it will be great to receive it :))You can find me on twitter at @LucianoPassos11Thanks for reading :)ReferencesSwiftSyntax Repo.
https://github.
com/apple/swift-syntaxapple/swift/lib/Syntax Docs and examples.
https://github.
com/apple/swift/tree/master/lib/Syntaxtry!.Swift NYC 2017 — Improving Swift Tools with libSyntax.
https://www.
youtube.
com/watch?v=5ivuYGxW_3MSwift Forums.
https://forums.
swift.
org.. More details