Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[flutter_tools] handle case where file is deleted by other program or running on read only volume #66708

Merged
merged 10 commits into from Oct 6, 2020

Conversation

@jonahwilliams
Copy link
Member

@jonahwilliams jonahwilliams commented Sep 26, 2020

Description

I've seen some odd bugs in crash reporting where the tool fails to delete a file that it just checked does exist. It could be that a different program was deleting the file and the timing was just right. Be defensive to this and test on an OSError of 2 if it still exists. If it does, then it is probably pointing at a read-only volume

Fixes #57950

@jonahwilliams jonahwilliams changed the title [flutter_tools] handle case where file is deleted by other program [flutter_tools] handle case where file is deleted by other program or running on read only volume Sep 26, 2020
@@ -17,6 +17,40 @@ import 'signals.dart';
// the tool is killed by a signal.
export 'package:file/file.dart';

extension FileUtils on File {

This comment has been minimized.

@jonahwilliams

jonahwilliams Sep 28, 2020
Author Member

Not in love with the extension method, but I don't want to re-type everything as ErrorHandling file.

Maybe just a static helper method?

This comment has been minimized.

@jonahwilliams

jonahwilliams Oct 1, 2020
Author Member

Update to static method on ErrorHandlingFileSystem

@jonahwilliams jonahwilliams requested a review from zanderso Sep 28, 2020
Copy link
Member

@zanderso zanderso left a comment

Extension methods are a bit scary. Not saying it's the not the right thing here, but it looks like we need to be careful with them.

@@ -17,6 +17,40 @@ import 'signals.dart';
// the tool is killed by a signal.
export 'package:file/file.dart';

extension FileUtils on File {

This comment has been minimized.

@zanderso

zanderso Sep 29, 2020
Member

Does this extension apply to every class that extends or implements File, or does it apply only to exactly File? Either way, it is probably worth a comment here.

This comment has been minimized.

@jonahwilliams

jonahwilliams Oct 1, 2020
Author Member

Moved to ErrorHandlingFileSystem

@@ -57,6 +58,44 @@ void main() {
});
});

testWithoutContext('deleteIfExists does not delete if file does not exist', () {

This comment has been minimized.

@zanderso

zanderso Sep 29, 2020
Member

If the extension method applies to any implementation of File, then we need test coverage of how it applies to them.

}
try {
deleteSync(recursive: recursive);
} on FileSystemException catch (err) {

This comment has been minimized.

@zanderso

zanderso Sep 29, 2020
Member

If File is an ErrorHandlingFile, then you could get either a ToolExit or an Exception here depending on whether you're under a noExitOnFailure.

This comment has been minimized.

@jonahwilliams

jonahwilliams Sep 29, 2020
Author Member

The goal with this catch is to only intercept error code 2, which we don't catch with error handling io because it normally seems to indicate a tool error.

Any other exception types should pass through as is, though I will add support for the existing error ignoring logic

rethrow;
}
if (existsSync()) {
throwToolExit(

This comment has been minimized.

@zanderso

zanderso Sep 29, 2020
Member

This would subvert a noExitOnFailure.

This comment has been minimized.

@jonahwilliams

jonahwilliams Sep 29, 2020
Author Member

Thats true - this should probably become a static method on error handling file system - then that can be taken into account.

This comment has been minimized.

@jonahwilliams

jonahwilliams Oct 1, 2020
Author Member

Updated to support noExitOnFailure

@jonahwilliams jonahwilliams requested a review from zanderso Oct 1, 2020
@jonahwilliams
Copy link
Member Author

@jonahwilliams jonahwilliams commented Oct 5, 2020

Friendly ping @zanderso

Copy link
Member

@zanderso zanderso left a comment

Sorry for the delay.

// been deleted by a different program while the tool was running.
// if it still exists, the file likely exists on a read-only volume.
// This error code is the same across linux, windows, and macOS.
const int kSystemCannotFindFile = 2;

This comment has been minimized.

@zanderso

zanderso Oct 5, 2020
Member

Wouldn't this be different on different OS's?

This comment has been minimized.

@jonahwilliams

jonahwilliams Oct 5, 2020
Author Member

For files it is the same, windows 2 is ERROR_FILE_NOT_FOUND, and 2 on linux/macOS 2 is ENOENT is No such file or directory. I can check if deleting a directory that does not exist uses the same error code on windows

This comment has been minimized.

@zanderso

zanderso Oct 5, 2020
Member

kk. This would be good to have as a comment in the code here.

This comment has been minimized.

@jonahwilliams

jonahwilliams Oct 5, 2020
Author Member

Made the comment more specific

/// This method should be prefered to checking if it exists and
/// then deleting, because it handles the edge case where the file is
/// deleted by a different program between the two calls.
static bool deleteFileIfExists(File file, {bool recursive = false}) {

This comment has been minimized.

@zanderso

zanderso Oct 5, 2020
Member

How about:

static bool deleteIfExists(FileSystemEntity entity, ...) { ... }

Then it can handle directories and links as well.

This comment has been minimized.

@jonahwilliams

jonahwilliams Oct 5, 2020
Author Member

Will check if these have the same error code on windows

This comment has been minimized.

@jonahwilliams

jonahwilliams Oct 5, 2020
Author Member

same error code for directories

@@ -241,9 +242,7 @@ flutter_project:lib/
toolExit: true,
);
} finally {
if (optionsFile.existsSync()) {

This comment has been minimized.

@zanderso

zanderso Oct 5, 2020
Member

How about replacing https://github.com/flutter/flutter/blob/master/packages/flutter_tools/test/src/common.dart#L34 and using that not just for directories but also for files?

This comment has been minimized.

@jonahwilliams

jonahwilliams Oct 5, 2020
Author Member

That probably needs to be fixed too, but it is more for file lock issues on windows than read-only problems

This comment has been minimized.

@zanderso

zanderso Oct 5, 2020
Member

So, the difference would be a different error code? The same code structure could handle the situation apart from that, though, right? Maybe not for this PR, but it would be nice to have a single way for the tool to try to delete something.

This comment has been minimized.

@jonahwilliams

jonahwilliams Oct 5, 2020
Author Member

I think we would prefer if the tests shut down cleanly, and this was added as more of a stopgap

This comment has been minimized.

@zanderso

zanderso Oct 5, 2020
Member

Ah, yeah. Good point.

Copy link
Member

@zanderso zanderso left a comment

lgtm

@@ -241,9 +242,7 @@ flutter_project:lib/
toolExit: true,
);
} finally {
if (optionsFile.existsSync()) {

This comment has been minimized.

@zanderso

zanderso Oct 5, 2020
Member

Ah, yeah. Good point.

@jonahwilliams jonahwilliams merged commit c8466d0 into flutter:master Oct 6, 2020
24 of 25 checks passed
24 of 25 checks passed
@flutter-dashboard
flutter-gold Pending.
Details
@flutter-dashboard
Linux analyze
Details
@flutter-dashboard
Linux build_tests
Details
@flutter-dashboard
Linux customer_testing
Details
@flutter-dashboard
Linux firebase_abstract_method_smoke_test
Details
@flutter-dashboard
Linux firebase_android_embedding_v2_smoke_test
Details
@flutter-dashboard
Linux firebase_release_smoke_test
Details
@flutter-dashboard
Linux fuchsia_precache
Details
@flutter-dashboard
Linux gradle_r8_test
Details
@flutter-dashboard
Linux tool_tests
Details
@flutter-dashboard
Linux web_integration_tests
Details
@flutter-dashboard
Mac customer_testing
Details
@flutter-dashboard
Mac tool_tests
Details
@wip
WIP Ready for review
Details
@flutter-dashboard
Windows customer_testing
Details
@flutter-dashboard
Windows tool_tests
Details
@cirrus-ci
analyze-linux Task Summary
Details
@googlebot
cla/google All necessary CLAs are signed
@cirrus-ci
customer_testing-linux Task Summary
Details
@cirrus-ci
deploy_gallery-linux Task Summary
Details
@cirrus-ci
deploy_gallery-macos Task Summary
Details
@flutter-dashboard
flutter-build
Details
@cirrus-ci
tool_tests-commands-linux Task Summary
Details
@cirrus-ci
tool_tests-general-linux Task Summary
Details
@cirrus-ci
web_integration_tests Task Summary
Details
@jonahwilliams jonahwilliams deleted the jonahwilliams:handle_external_delete branch Oct 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants