Skip to content

Move to fat pointers to remove UB#2382

Closed
carbotaniuman wants to merge 10 commits into
RustPython:mainfrom
carbotaniuman:fat-ptr
Closed

Move to fat pointers to remove UB#2382
carbotaniuman wants to merge 10 commits into
RustPython:mainfrom
carbotaniuman:fat-ptr

Conversation

@carbotaniuman
Copy link
Copy Markdown
Contributor

@carbotaniuman carbotaniuman commented Dec 26, 2020

closes #2381

@coolreader18
Copy link
Copy Markdown
Member

Is there any way to keep the vtable inside PyInner and still have the drop_rc/drop_weak? It's a lot more efficient to have PyObjectRef be pointer-sized, since that halves the memory usage for a ton of data structures that we use. I think you should be able to do let drop_rc = self.inner.vtable.drop_rc at the top of fn drop, and then call it later

@carbotaniuman
Copy link
Copy Markdown
Contributor Author

It doesn't work for Weak pointers, unless we use something like what CPython does with a linked list.

@coolreader18
Copy link
Copy Markdown
Member

Oh, true... hmm

@coolreader18
Copy link
Copy Markdown
Member

Would it be possible to only make PyObjectWeak a fat pointer? Store it in PyObject, and then copy the vtable reference when we want to make a weak?

@carbotaniuman
Copy link
Copy Markdown
Contributor Author

The other problem is that I get stack overflows again :(, gonna see what is causing those. Making PyObjectWeak a fat pointer might fix that.

@coolreader18
Copy link
Copy Markdown
Member

I might also look into unifying the vtable and PyTypeRef, since CPython has the tp_dealloc slot and all, I just have to figure out how to make that safe still to create types

@carbotaniuman
Copy link
Copy Markdown
Contributor Author

Hmmm more stack overflows. I put it in LLDB and it keeps calling the same destructor over and over... Weird.

@carbotaniuman
Copy link
Copy Markdown
Contributor Author

Do you remember what the WASM bug was last time?

@coolreader18
Copy link
Copy Markdown
Member

coolreader18 commented Dec 27, 2020

It looks like you figured it out, but I had made layout assumptions that didn't hold up on wasm, specifically wrt struct Inner { vtable, obj: PyObject<T> } vs just adding the vtable field to PyObject IIRC. The first one added extra padding on wasm that made the 2 options not equivalent, but I was assuming that they were.

@dralley
Copy link
Copy Markdown
Contributor

dralley commented Dec 28, 2020

This appears to fix #2381 FYI, if you want to add closes #2381 to your commit message.

edit: I guess it would be more accurate to say that "instead of immediately exiting and complaining at me, miri instead slowly heats up my CPU for >15 minutes while it chews through my entire 32gb of RAM". Miri is slow AF. I guess it's still a good sign.

@youknowone
Copy link
Copy Markdown
Member

@coolreader18 @carbotaniuman how is this ended up?

@fanninpm
Copy link
Copy Markdown
Contributor

Now that the structure of pyobjectrc.rs has completely changed, does it make sense to keep this PR open as it currently stands?

@coolreader18
Copy link
Copy Markdown
Member

FYI I'm finally working on a big overhaul to PyObjectRef's implementation/our object model that'll fix the UB: coolreader18:big-object-model-overhaul

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Miri catches some undefined behavior

5 participants