Software development gets easier when you have access to the tools you need to do your best work. With simple, cohesive workflows, you can reduce context switching fatigue, streamline authentication processes, and focus on problems that really matter. To help you get to a seamless developer experience, we’re continuing our partnership with Microsoft to bring Azure Pipelines into GitHub. This new CI/CD service enables you to continuously build, test, and deploy to any platform, and it’s free for open source repositories.

Azure Pipelines in Marketplace

Azure Pipelines is now available in GitHub Marketplace. With this new integration, you can easily configure a CI/CD pipeline for any Azure application using your preferred language and framework as part of your GitHub workflow in just a few simple steps.

Build and deploy with ease

Automatically trigger your pipeline with changes to your repository when you connect Azure Pipelines with GitHub. Get rich status reports, annotated code, and detailed information—all within the GitHub interface.

Works with any platform

Build, test, and deploy applications to virtual machines; to cloud providers such as Azure, Amazon Web Services, and Google Cloud Platform; or to app stores across operating systems, including Android, iOS, Linux, macOS, and Windows systems. You can even take advantage of built-in tasks for Kubernetes, serverless, and VM deployments as well as a rich ecosystem of extensions for every language and tool.

Free for open source projects

Many open source repositories are already using Azure Pipelines for CI/CD, such as CPython, webpack, Visual Studio Code, TypeScript, Atom, and Electron. Get up and running quickly with unlimited build minutes, and up to 10 free parallel Microsoft-hosted build jobs across Linux, macOS, or Windows.

We’re constantly working towards building an ecosystem that values openness, provides equal opportunities to partners, and empowers developers with access to the right tools. While this integration offers improvements to the way you build with GitHub and Azure Pipelines, we know everyone has their own approach to CI. View a list of our CI partners in GitHub Marketplace to find a solution that works for you.

Start using Azure Pipelines

Interested in building on our platform? Contact us or visit GitHub Developer for more information.

The open source Git project just released Git 2.19, with features and bug-fixes from over 60 contributors. Here’s a look at some of the most interesting features introduced in the latest versions of Git.

Compare histories with git range-diff

You might have used git rebase, which is a powerful tool for rewriting history by altering commits, commit order, or branch bases to name a few. Many people do this to “polish” a series of commits before proposing to merge them into a project. But how can we visualize the differences between two sets of commits, before and after a rebase?

We can use git diff to show the difference between the two end states, but that doesn’t provide information about the individual commits. And if the base on which the commits were built has changed, the resulting state might be quite different, even if the changes in the commits are largely the same.

Git 2.19 introduces git range-diff, a tool for comparing two sequences of commits, including changes to their order, commit messages, and the actual content changes they introduce.

In this example, we rewrote a series of three commits, and compared the tips of each version using git range-diff. git range-diff shows that we moved the commit introducing README.md to be first instead of second, amended both the commit message and body of the typo fix, and introduced a new commit to add a missing newline.

[source]

git grep’s new tricks

When you search for a phrase using git grep, it’s often helpful to have additional information pertaining to each match, such as its line number and function context.

In Git 2.19 you can now locate the first matching column of your query with git grep --column.

If you’re using Vim, you can also try out git-jump, a Git add-on that converts useful locations in your code to jump locations in your text editor. git-jump can take you to merge conflicts, diff hunks, and now, exact grep locations with git grep --column.

git grep also learned the new -o option (meaning --only-matching). This is useful if you have a non-trivial regular expression and want to gather only the matching parts of your search.

For example, if you want to count all of the various ways that the Git source code spells “SHA-1” (e.g., “sha1”, “SHA1”, and so on):

(The other options -hiI are to omit the filename, search case-insensitively, and ignore matches in binary files, respectively.)

[source, source]

Sorting branches

The git branch command, like git tag (and their scriptable counterpart, git for-each-ref), takes a --sort option to let you order the results by a number of properties. For example, to show branches in the order of most recent update, you could use git branch --sort=-authordate. But if you always prefer that order, typing that sort option can get tiresome.

Now, you can use the branch.sort config to set the default ordering of git branch:

Note that by default, git branch sorts by refname, hence master is first and newest is last. In the above example, we tell Git that we would instead prefer the most recently updated branch first, and the rest in descending order. Hence, newest is first and master is last.

You might also want to try these other sorting options:

• --sort=numparent shows merges by how awesome they are
• --sort=refname sorts branches alphabetically by their name (this is the default, but may be useful to override in your configuration)
• --sort=upstream sorts branches by the remote from which they originate

[source]

Directory rename detection

Git has always detected renamed files as part of merges. For example, if one branch moves a file from A to B and another modifies content in A, then the resulting merge will apply that modification to the content’s new location in B.

The same thing can happen with files in a directory. If one branch moves a directory from A to B but another adds a new file A/file, we can infer that the file should become B/file when the two are merged. In Git 2.18, git merge does this whenever rename detection is enabled (which is by default).

[source]

Tidbits

• In Git v2.18, a remote code execution vulnerability in .gitmodules was fixed, where an attacker could execute scripts when the victim cloned with --recurse-submodules. If you haven’t upgraded, please do! The fix was also backported to v2.17.1, v2.16.4, v2.15.2, v2.14.4, and v2.13.7, so you’re safe if you’re running one of those. [source]

• Have you ever run into a Git command line option that should have tab-completed but didn’t? Keeping these up to date has long been an annoying source of manual work for the project, but now the completion of options for most commands is generated automatically (along with the list of commands itself, the names of config options, and more). [source, source, source, source]

• gpg signing and verification of commits and tags has been extended to work with gpgsm, which uses X.509 certificates instead of OpenPGP keys. These certificates may be easier to manage for centralized groups (e.g., developers working for a large enterprise). [source]

• To fetch a configuration variable with a “fallback” value, it’s common for scripts to say git config core.myFoo || echo <default>. But that doesn’t give Git the opportunity to interpret <default> for you. When it comes to colors, this is especially important for instances where you ultimately need the ANSI color code, for say, “bold red”, but don’t want to type \033[1;31m.

  git config has long supported this with a special --get-color option, but now there are options that can be applied uniformly to all types of config. For instance, git config --type=int --default=2M core.myInt will expand the default to 2097152, and git config --type=expiry --default=2.weeks.ago gc.pruneExpire consistently returns a number of seconds. [source, source]

• Quick quiz: if git tag -l is shorthand for git tag --list, then what does git branch -l do? If you thought, “surely it doesn’t list all branches”, then congratulations: you’re a veteran Git user!

  In fact, git branch -l has been used since 2006 to establish a reflog for a newly created branch, something that you probably didn’t care about since it became the default shortly after being introduced.

  That usage has been deprecated (you will receive a warning if you use git branch -l), thus clearing the way for git branch -l to mean git branch --list. [source]

• In our last post, we discussed the new --color-moved option, which (unsurprisingly) colors lines moved in a diff. The lines that were moved must be identical, meaning that the feature would miss re-indented code unless you specified a diff option such as --ignore-space-change. Keep in mind that this option would affect the whole diff, potentially missing space changes that you do care about. In Git 2.19, the whitespace for move detection can be configured independently with the new --color-moved-ws option. [source]

• Many of Git’s commands are colorized, like git diff, git status, and so on. Since 2.17, a few more commands improved their support for colorization, too. git blame learned to colorize lines based on age or by group. Messages sent from a remote server are now colorized based on their keyword (e.g., “error”, “warning”, etc.). Finally, push errors are now painted red for increased visibility. [source, source, source]

• If you’ve ever run git checkout with the name of a remote branch, you might know that Git will automatically create a local branch that tracks the remote one. However, if that branch name is found in more than one remote, Git does not know which to use, and simply gives up.

  In 2.19, Git learned the checkout.defaultRemote configuration, which specifies a remote to default to when resolving such an ambiguity. [source]

• Git interprets certain text encodings (e.g. UTF-16) as binary, meaning that tools like git diff will not show a textual diff. Normally it’s recommended to store your text files as UTF-8, but this isn’t always possible if other tools generate or expect another encoding.

  You can now tell Git which encoding you prefer in your working tree on a per-file basis by setting the working-tree-encoding attribute. This will cause Git to store the files as UTF-8 internally, and convert them back to your preferred encoding on checkout. The result looks good in git diff, as well as on hosting sites. [source]

Cooking

Some features are so big that they’re developed over the course of several releases. We have historically avoided reporting on works in progress in these posts, since the features are often still experimental, or there’s nothing you can directly start using.

That said, some of the topics upstream around this release are too exciting to ignore! So, here’s an incomplete summary of what’s happening upstream:

Partial clones

An important part of Git’s decentralized design is that all clones receive the full history of the project, making all clones true peers of one another. When there aren’t a large number of objects in your repository, things go quickly, but at a certain size clones can become frustratingly slow.

There’s ongoing work to allow “partial” clones which omit some blob and tree objects, in favor of requesting objects from the server as-needed. You can see a design overview of the feature, or even start experimenting yourself. Note that most public servers do not yet support the feature, but you can play with git clone --filter=blob:none against your local Git 2.19 install.

[source, source, source, source, source, source]

Commit graphs

Git has a very simple data model: everything is an object named after the hash of its contents, and objects point to each other by those names. Many operations walk the graph formed by those pointers. For example, asking “which releases contain this bug-fix” is really “which tag objects have a path to walk back to commit X” (where X is the commit fixing the aforementioned bug).

Those walks have traditionally required loading each object from disk to find its pointers. But now Git can compute and store properties of each commit in a more efficient format, leading to significantly faster traversals. You can read more about it in a series of blog posts from the feature’s author.

[source, source, source]

Protocol v2

Git still uses roughly the same protocol for fetching that was developed in 2005: after a client connects, the server dumps the current state of all branches and tags (called the “ref advertisement”), and then the client asks for the parts it needs to update. As repositories have grown, the cost of this advertisement has become a source of inefficiency.

The protocol has added new features over the years in a backwards-compatible way by negotiating capabilities between the server and client. But one thing that couldn’t be changed is the ref advertisement itself, because it happens before there’s a chance to negotiate.

Now there’s a new protocol which addresses this (and more), providing a way to transfer the advertisement more efficiently. Only a few servers support the new protocol so far, but you can read more about it in this blog post from its designer.

[source, source, source, source]

Transitioning away from SHA-1

We mentioned earlier that all Git objects are named according to a hash of their contents. You might know that the algorithm that determines the value of that hash is SHA-1, which has not been considered safe for some time. In fact, a collision attack was discovered and published last year, which we wrote about in our post on its remediation.

Though SHA-1 collisions in Git are unlikely in practice, the Git project has decided to pick a new hashing algorithm and has made significant progress towards implementing it. Git has chosen SHA-256 as the successor to SHA-1, and is working through the transition plan to convert to it.

[source]

And everything else

That’s just a sampling of changes from the last few versions. Read the full release notes for 2.19, or find the release notes for previous versions in the Git repository.

On September 12 the European Parliament will vote on amendments to the EU Copyright Directive, which will greatly impact the future of open source, European competitiveness, and software development in general. We urge you in the EU to contact your Members of European Parliament (MEPs) to tell them how important open source is to all software development and to the EU. Check out our previous post for background and talking points.

Read on for details on the implications for software and society, key amendments being voted on, and how you can make a difference.

On September 5, we hosted HackerOne, Wikimedia, Reddit, and the Electronic Frontier Foundation (EFF) at our San Francisco office for the event How developers can defend open source from the EU copyright proposal, addressing the many EU developers working in the Bay Area.

We kicked off the event with our own Julio Avalos giving a big-picture look at where this proposal fits into the tech policy landscape. We explained that the copyright proposal would affect developers by requiring upload filters (Article 13), imposing a “link tax” (Article 11), and leaving text and data mining restricted (Article 3). Mårten Mickos, CEO of HackerOne, emphasized the proposal’s impacts on open source software for HackerOne and in the EU. Then, we moderated a panel with Wikimedia Foundation’s Senior Public Policy Manager, Jan Gerlach, Reddit’s Director of Policy, Jessica Ashooh, and EFF’s International Director, Danny O’Brien covering their communities’ involvement in advocacy and their thoughts on future implications of the proposal.

For example, recognizing that copyright law in many ways hasn’t kept up with the digital age, Wikimedia identified priorities for copyright reform, including protecting the public domain and freedom of panorama, as well as allowing sufficiently broad exceptions to copyright for user-generated content and for text and data mining. Learn more on their Fix Copyright landing page.

Our call to action at the event was the same as it is here: Developers, tell your MEPs to protect software.

What Parliament will decide on September 12

So what exactly is Parliament voting on? The September 12 vote is not a simple yes or no—it’s actually quite complex. MEPs will vote on a number of amendments to the full directive that the EU Commission proposed two years ago. From the perspective of protecting software development, we offered our thoughts on what developers could tell MEPs that might be useful.

Details about key amendments MEPs will consider:

For an open internet

• The IMCO/LIBE committee amendments are most effective in promoting openness by focusing on active platforms and only requiring licensing agreements, not upload filters.
• The latest European People’s Party (EPP) (MEP Voss) amendments no longer talk about “measures” (which, in previous drafts, meant upload filters) but would still require filters unless platforms obtain licenses for copyrighted content.

For software development

• Article 13 (based on Article 2’s definition of what is in scope)
  • MEP Schaake’s amendments are the most protective of software development by more narrowly defining who is in scope for Article 13 (platforms where people upload music or video) and more broadly excluding software (archives and repositories, along with open source software development platforms).
  • The JURI Committee, ALDE party and latest EPP amendments all contain the same exclusion for “open source software developing platforms.”
• Article 3
  • Both the IMCO/LIBE and Schaake amendments benefit developers by creating exceptions for text and data mining (TDM) wherever the beneficiary has lawful access (rather than only for research organizations conducting TDM on a not-for-profit basis).
• Article 11
  • The IMCO/LIBE amendments also protect developers by prohibiting a link tax (meaning a press publisher cannot require a license to use a link needed to identify or request a source’s contents). They also give press publishers named in a publication the ability to conclude licensing agreements, rather than granting them rights by default.

While there’s a lot to be concerned about in the Copyright Directive, it’s important to recognize the need for positive copyright reform that reflects the digital world developers are creating. Some amendments reflect this reality, like articles on freedom of panorama and exclusion of user-generated content, which are in the IMCO/LIBE and Schaake amendments.

There’s still time to contact your MEPs before they vote on Wednesday! Developers have an important role to play in explaining how software works and what’s at stake. Contact us if you need more information about the EU Copyright Directive.

There’s a new way to view and interact with new pull requests in Atom through the GitHub package!

@smashwilson and @simurai have been working on a number of improvements that we’re excited to share with you today, with more on the horizon.

New pull request list view

First, we’ve added a pull request list view. Now you can see the most recent pull requests in the GitHub tab Ctrl+8 with information such as:

• Author’s avatar
• Title of the pull request
• Pull request number
• CI status
• “Last Updated” information

When you click on a pull request, you’ll see a view similar to the conversation view on GitHub.com, and can quickly check out the PR with the click of a button!

The top of the view contains the most important information:

• The pull request author (clicks to their GitHub.com profile)
• The pull request title
• The status of the pull request (for example, open)
• The organization, repository, and pull request number (clicks to the pull request on GitHub.com)
• A Checkout button that allows you to quickly checkout this pull request inside of Atom
• A Refresh button to pull updated information about the pull request
• Your CI and build status, with quick links to view the details
• The description of the pull request
• Reactions (currently unclickable)

After this information, you will find the same conversation view you would see on GitHub.com. Sometimes, it may be something as simple as the list of commits and other times it might be an entire conversation.

Hover card functionality

We’ve also added some user experience enhancements, such as enabling the hover card functionality in @mentions and references to other issues and pull requests:

Creating pull requests

In addition to seeing more information about your existing pull requests, you can also open a new pull request directly from Atom.

Starting from master, you will see a new message in the GitHub pane providing you with information on what you might want to do next, such as checking out an existing branch or creating a new branch. If you create a new branch, you will be prompted to start making changes to your branch. Finally, if you make changes, stage them, and commit to your branch, you will be invited to publish your branch and create a pull request with those changes:

Clicking Publish + open a new pull request will launch your browser at the draft of your pull request on GitHub.com. There, you can add an extensive description, reviewers, labels, and more. Visit https://github.atom.io/ for more information on the GitHub panel in Atom.

Emoji support in commit messages

We also care about making the experience consistent with GitHub.com. You might notice that commit messages in Atom now support emoji! :sparkles: to @annthurium for making committing in Atom a bit more entertaining:

Learning from our users

We’re excited about the new experiences we’re bringing to the Atom community and looking forward to continuing to improve our package. You might have seen in a recent blog post that we’re working on improving our understanding of who you are, how you write code, and how you collaborate with your team. This involves usability studies, as well as a large project that @annthurium and @jasonrudolph have been working on to improve our metric gathering. Read about the details of Telemetry on the Atom Blog.

We have integrated Telemetry into our GitHub package for Atom so that we can better understand what features are useful—and which are being left undiscovered. We invite you to revisit your opt-in decision on metrics if you’re interested in helping us improve our package by sending metrics through our secure GitHub data pipeline. Just open your Atom Preferences and choose Allow limited anonymous usage states, exception, and crash reporting.

Try GitHub for Atom

It’s been two years since the first cohort of student leaders became GitHub Campus Experts. In that time, more than 100 Campus Experts in 20 countries have made their campuses a better place to learn about technology. As GitHub Interns, they have built tech communities in Africa, organized inclusive conferences, and shipped amazing projects.

Now it’s your turn! If you’re a student—age 18 or older—using the Student Developer Pack, you’re eligible to apply for the next class of Campus Experts. Once accepted, you can access the Campus Experts training and develop skills in public speaking, workshop design, and community management. You will also receive training and mentorship from GitHub employees, opportunities to participate in exclusive events, and support to help you grow the developer community at your school.

Apply now