Is sort() stable in Swift 5?

Martin · 2019-03-06T19:24:46.373Z

The documentation of the current Sort.swift still states that

  /// The sorting algorithm is not guaranteed to be stable

On the other hand, the actual implementation calls

  /// Sorts the elements of this buffer according to `areInIncreasingOrder`,
  /// using a stable, adaptive merge sort.
  ///
  /// The adaptive algorithm used is Timsort, modified to perform a straight
  /// merge of the elements using a temporary buffer.
  @inlinable
  public mutating func _stableSortImpl(
    by areInIncreasingOrder: (Element, Element) throws -> Bool
  ) rethrows { ... }

Does that mean that sort() is stable in Swift 5? I could not find that documented in the CHANGELOG or in an evolution proposal.

Timsort is mentioned as a possible choice for a stable sort algorithm in [Starter Proposal] Add Stable Sort to StdLib, but that discussion ended in March '18.

There is also SR-306 Add stable sort algorithm, which is still in the Open state.

Regards, Martin

xwu · 2019-03-06T19:31:36.468Z

If I recall, sort() is currently stable, but it is not yet guaranteed to be stable (meaning, the fact that it is stable is currently an implementation detail, and a future version of Swift could ship an unstable algorithm instead).

tkrajacic · 2019-03-06T19:33:05.968Z

This makes me think it would actually be beneficial to make it intentionally unstable. Kinda like the hashValue randomization. So erroneously relying on it to be stable because not reading the documentation would surface this error.

xwu · 2019-03-06T19:34:47.847Z

tkrajacic:

This makes me think it would actually be beneficial to make it intentionally unstable. Kinda like the hashValue randomization. So erroneously relying on it to be stable because not reading the documentation would surface this error.

The algorithm was only recently made stable in preparation for a proposal to make it guaranteed as such.

Martin · 2019-03-06T19:44:25.067Z

Thank you for the clarification. I did now look up the relevant commit:

[stdlib] Switch to a stable sort algorithm (#19717)

This switches the standard library's sort algorithm from an in-place introsort to use a modified timsort, a stable, adaptive sort that merges runs using a temporary buffer. This implementation performs straight merges instead of adopting timsort's galloping strategy.

which indeed changed the comment

/// The sorting algorithm is not stable.

to

/// The sorting algorithm is not guaranteed to be stable.

ben-cohen · 2019-03-06T19:49:23.803Z

xwu:

The algorithm was only recently made stable in preparation for a proposal to make it guaranteed as such.

Exactly. It was very important that this change happened before Swift shipped in an OS. Otherwise, whether or not it was stable would be determined by which version of the operating system you are running on, which would not be tenable. But since the version we're shipping in 5.0 is stable, we still have the option of officially locking that in as a guarantee in future.

Note, the new stable sort was, on balance, significantly faster in most cases than the old unstable sort so we didn't give anything up in exchange for reserving this future possibility :)

hisekaldma · 2019-07-02T09:35:17.314Z

Any news on locking down the sort to always be stable?

AlexSedykh · 2019-07-04T09:21:58.279Z

I would like to know. Does these sorts algorithms (in Standard Library) use some of advanced sorts approaches, like bubble sort, merge sort, bucket sort e.t.c in their internal implementations.

ben-cohen · 2019-07-04T15:03:00.984Z

The library currently uses a timsort.

dabrahams · 2019-07-19T23:28:30.574Z

ben-cohen:

Exactly. It was very important that this change happened before Swift shipped in an OS. Otherwise, whether or not it was stable would be determined by which version of the operating system you are running on, which would not be tenable.

If such a change is considered untenable, shouldn't we lock in stability now?

ben-cohen · 2019-07-22T02:37:34.296Z

It depends what you mean by "lock in". Any change to the algorithm would need to be approved by the code owner, and a PR to make it unstable would close off the ABI to it ever being stable which isn't a PR we'd merge, so there's no risk of it "accidentally" becoming unstable. We're now at a point where someone could propose either to officially make sort stable or unstable, but either change would need to go through evolution.

DaveZ · 2019-07-22T07:05:58.580Z

I understand why a stable sort is ABI, but why would an unstable sort ABI? If the result happen to be stable later, why would that break anything?

ben-cohen · 2019-07-22T15:14:09.534Z

DaveZ:

If the result happen to be stable later, why would that break anything?

This would require that if you rely on the stability of the sort, you should only do so within an availability guard, otherwise whether or not it was stable would depend on the deployment target. But because it's the same symbol, the compiler won't tell you this, you just have to know/remember, leading to some pretty unpleasant bug potential.

ben-cohen · 2019-07-22T15:17:21.771Z

I guess I'm using to strong a phrasing when I say "close off the ABI to it ever being stable". It would make relying on its stability going forward very tricky in a way that's related to the ABI having been declared because the algorithm now ships in the OS.

DaveZ · 2019-07-23T08:17:07.212Z

Okay. I see your point. I think/hope that reasonable people can agree that "ABI compatibility" is a spectrum. There was a time when ABI compatibility for an OS/library meant only backward compatibility, therefore having sort be unstable in an older release and stable in a newer release would be considered reasonable. That being said, I think you're right, the definition of ABI compatibility (at least for Apple) now requires that library authors consider backwards deployment from a recent SDK as ABI, and therefore sort cannot be changed to a stable algorithm.

Even if we ignore ABI, there is an entirely different but principled argument that an essential API like "sort()" should minimize surprising behavior, therefore it should be stable. If people want to opt into an unstable sort or any other tuning parameter, then let them call "unstableSort()" or "sort(tuningParam:)", etc.

justinmeiners · 2022-07-07T21:18:33.769Z

Hoping to bring attention to this issue again. I would be interested in stability being required for the implementation.

xwu · 2022-07-07T22:25:55.614Z

Assigning a dictionary without re-hashing Using Swift

Absolutely. This is a (semantic) API change, so I expect it'll require going through Swift Evolution. I've been collecting other ill-defined aspects of the stdlib -- perhaps we have a large enough batch of these now to be worth the expense of a small proposal. (Edit: unifying and documenting the expected behavior of base properties on mutable slices could also be part of this, although that one also requires actual behavioral changes in some obscure corners of the stdlib.)