Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add base support for exporting objects and classes in node_loader #343

Draft
wants to merge 12 commits into
base: develop
Choose a base branch
from

Conversation

rxbryan
Copy link
Contributor

@rxbryan rxbryan commented Sep 15, 2022

Description

add base support for exporting objects and classes in node_loader

Fixes #(issue_no)

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update
  • Documentation update

Checklist:

  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have made corresponding changes to the documentation.
  • My changes generate no new warnings.
  • I have added tests/screenshots (if any) that prove my fix is effective or that my feature works.
  • I have tested the tests implicated (if any) by my own code and they pass (make test or ctest -VV -R <test-name>).
  • If my change is significant or breaking, I have passed all tests with ./docker-compose.sh build &> output and attached the output.
  • I have tested my code with OPTION_BUILD_SANITIZER or ./docker-compose.sh test &> output and OPTION_TEST_MEMORYCHECK.
  • I have tested with Helgrind in case my code works with threading.
  • I have run make clang-format in order to format my code and my code follows the style guidelines.

If you are unclear about any of the above checks, have a look at our documentation here.

@viferga
Copy link
Member

viferga commented Feb 7, 2023

Last commits look pretty well, I recommend you to implement tests in order to verify the behavior. TDD is always easier to develop. Check out metacall-java-test for having an example of how to implement the test.

@rxbryan rxbryan marked this pull request as draft February 14, 2023 16:16
@viferga
Copy link
Member

viferga commented Feb 15, 2023

Check out the tests, no one is passing, try to pass them before pushing.

@rxbryan rxbryan force-pushed the node-feat1 branch 2 times, most recently from e44e0ab to f6078f4 Compare February 20, 2023 10:33
@rxbryan
Copy link
Contributor Author

rxbryan commented Feb 20, 2023

Check out the tests, no one is passing, try to pass them before pushing.

I just figured out what's making the tests fail. I would fix it as soon as possible

add api for getting all keys and values in adt_set and adt_map
@rxbryan rxbryan marked this pull request as ready for review February 21, 2023 07:55
@viferga
Copy link
Member

viferga commented Feb 21, 2023

I node port it seems there's a bug:

2023-02-21T01:41:33.6835391Z [Tue Feb 21 00:11:52] #13550 [ 4174 | py_loader_impl_gc_print | /usr/local/metacall/source/loaders/py_loader/source/py_loader_impl.c ] @debug : Python Garbage Collector:
2023-02-21T01:41:33.6835405Z
2023-02-21T01:41:33.6835481Z ==13543==T8 TSDDtor
2023-02-21T01:41:33.6835553Z ==13543==T8 exited
2023-02-21T01:41:33.6835560Z
2023-02-21T01:41:33.6835696Z attempt to reference unallocated memory (signal SIGSEGV)

@viferga
Copy link
Member

viferga commented Feb 21, 2023

Also there's this in windows:

2023-02-21T00:03:25.9422754Z ==1552==ERROR: AddressSanitizer: access-violation on unknown address 0x7ff86ac92a68 (pc 0x7ff86ac92a68 bp 0x00000000000f sp 0x0040fcbffa18 T2)
2023-02-21T00:03:25.9422849Z #0 0x7ff86ac92a67 ()
2023-02-21T00:03:25.9422997Z #1 0x7ff89bfa5f24 (C:\Windows\SYSTEM32\ntdll.dll+0x180075f24)
2023-02-21T00:03:25.9423135Z #2 0x7ff89bf532e2 (C:\Windows\SYSTEM32\ntdll.dll+0x1800232e2)
2023-02-21T00:03:25.9423277Z #3 0x7ff89bf8a3dd (C:\Windows\SYSTEM32\ntdll.dll+0x18005a3dd)
2023-02-21T00:03:25.9423433Z #4 0x7ff898798ca9 (C:\Windows\System32\KERNELBASE.dll+0x180078ca9)
2023-02-21T00:03:25.9423581Z #5 0x7ff898ae2793 (C:\Windows\System32\ucrtbase.dll+0x180022793)
2023-02-21T00:03:25.9423720Z #6 0x7ff898ae2690 (C:\Windows\System32\ucrtbase.dll+0x180022690)
2023-02-21T00:03:25.9424017Z #7 0x7ff86133f579 (C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64\clang_rt.asan_dbg_dynamic-x86_64.dll+0x18005f579)
2023-02-21T00:03:25.9424527Z #8 0x7ff89b607ad3 (C:\Windows\System32\KERNEL32.DLL+0x180017ad3)
2023-02-21T00:03:25.9424654Z #9 0x7ff89bf8a370 (C:\Windows\SYSTEM32\ntdll.dll+0x18005a370)
2023-02-21T00:03:25.9424665Z
2023-02-21T00:03:25.9424797Z AddressSanitizer can not provide additional info.
2023-02-21T00:03:25.9424947Z SUMMARY: AddressSanitizer: access-violation ()
2023-02-21T00:03:25.9425037Z Thread T2 created by T0 here:
2023-02-21T00:03:25.9425338Z #0 0x7ff861340e38 (C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64\clang_rt.asan_dbg_dynamic-x86_64.dll+0x180060e38)
2023-02-21T00:03:25.9425484Z #1 0x7ff898ae2356 (C:\Windows\System32\ucrtbase.dll+0x180022356)
2023-02-21T00:03:25.9425636Z #2 0x7ff85c4f9306 (D:\a\core\core\build\Debug\libnode.dll+0x180419306)
2023-02-21T00:03:25.9425792Z #3 0x7ff85c4f924b (D:\a\core\core\build\Debug\libnode.dll+0x18041924b)
2023-02-21T00:03:25.9426040Z #4 0x7ff8632c15d8 in node_loader_impl_initialize D:\a\core\core\source\loaders\node_loader\source\node_loader_impl.cpp:6577
2023-02-21T00:03:25.9426232Z #5 0x7ff866c26344 in loader_impl_initialize D:\a\core\core\source\loader\source\loader_impl.c:367
2023-02-21T00:03:25.9426433Z #6 0x7ff866c22093 in loader_impl_load_from_file D:\a\core\core\source\loader\source\loader_impl.c:826
2023-02-21T00:03:25.9426619Z #7 0x7ff866c1ea69 in loader_load_from_file D:\a\core\core\source\loader\source\loader.c:317
2023-02-21T00:03:25.9426811Z #8 0x7ff866c2bbc2 in metacall_load_from_file D:\a\core\core\source\metacall\source\metacall.c:348
2023-02-21T00:03:25.9427182Z #9 0x7ff7429f302d in metacall_node_event_loop_test_DefaultConstructor_Test::TestBody(void) D:\a\core\core\source\tests\metacall_node_async_resources_test\source\metacall_node_async_resources_test.cpp:45
2023-02-21T00:03:25.9427630Z #10 0x7ff742a2bc8c in testing::internal::HandleSehExceptionsInMethodIfSupported<class testing::Test, void>(class testing::Test , void (__cdecl testing::Test::)(void), char const *) D:\a\core\core\build\source\tests\src\google-test-depends\googletest\src\gtest.cc:2592
2023-02-21T00:03:25.9428053Z #11 0x7ff742a2b982 in testing::internal::HandleExceptionsInMethodIfSupported<class testing::Test, void>(class testing::Test , void (__cdecl testing::Test::)(void), char const *) D:\a\core\core\build\source\tests\src\google-test-depends\googletest\src\gtest.cc:2643
2023-02-21T00:03:25.9428367Z #12 0x7ff742a08eba in testing::Test::Run(void) D:\a\core\core\build\source\tests\src\google-test-depends\googletest\src\gtest.cc:2682
2023-02-21T00:03:25.9428626Z #13 0x7ff742a099c4 in testing::TestInfo::Run(void) D:\a\core\core\build\source\tests\src\google-test-depends\googletest\src\gtest.cc:2861
2023-02-21T00:03:25.9428886Z #14 0x7ff742a0a201 in testing::TestSuite::Run(void) D:\a\core\core\build\source\tests\src\google-test-depends\googletest\src\gtest.cc:3015
2023-02-21T00:03:25.9429169Z #15 0x7ff742a1082b in testing::internal::UnitTestImpl::RunAllTests(void) D:\a\core\core\build\source\tests\src\google-test-depends\googletest\src\gtest.cc:5855
2023-02-21T00:03:25.9429662Z #16 0x7ff742a2be1c in testing::internal::HandleSehExceptionsInMethodIfSupported<class testing::internal::UnitTestImpl, bool>(class testing::internal::UnitTestImpl , bool (__cdecl testing::internal::UnitTestImpl::)(void), char const *) D:\a\core\core\build\source\tests\src\google-test-depends\googletest\src\gtest.cc:2592
2023-02-21T00:03:25.9430150Z #17 0x7ff742a2bc02 in testing::internal::HandleExceptionsInMethodIfSupported<class testing::internal::UnitTestImpl, bool>(class testing::internal::UnitTestImpl , bool (__cdecl testing::internal::UnitTestImpl::)(void), char const *) D:\a\core\core\build\source\tests\src\google-test-depends\googletest\src\gtest.cc:2643
2023-02-21T00:03:25.9430398Z #18 0x7ff742a0a9d0 in testing::UnitTest::Run(void) D:\a\core\core\build\source\tests\src\google-test-depends\googletest\src\gtest.cc:5438
2023-02-21T00:03:25.9430714Z #19 0x7ff7429f29e0 in RUN_ALL_TESTS(void) D:\a\core\core\build\source\tests\src\google-test-depends\googletest\include\gtest\gtest.h:2490
2023-02-21T00:03:25.9430924Z #20 0x7ff7429f1850 in main D:\a\core\core\source\tests\metacall_node_async_resources_test\source\main.cpp:27
2023-02-21T00:03:25.9431127Z #21 0x7ff742a64f88 in invoke_main D:\a_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
2023-02-21T00:03:25.9431358Z #22 0x7ff742a64edd in __scrt_common_main_seh D:\a_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
2023-02-21T00:03:25.9431580Z #23 0x7ff742a64d9d in __scrt_common_main D:\a_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:330
2023-02-21T00:03:25.9431791Z #24 0x7ff742a64ffd in mainCRTStartup D:\a_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_main.cpp:16
2023-02-21T00:03:25.9431946Z #25 0x7ff89b607ad3 (C:\Windows\System32\KERNEL32.DLL+0x180017ad3)
2023-02-21T00:03:25.9432093Z #26 0x7ff89bf8a370 (C:\Windows\SYSTEM32\ntdll.dll+0x18005a370)

But I think this has been already solved in develop, can you merge develop into your branch and push so we can see if that test passes?

@rxbryan
Copy link
Contributor Author

rxbryan commented Feb 21, 2023

Also there's this in windows:

But I think this has been already solved in develop, can you merge develop into your branch and push so we can see if that test passes?

My branch is up-to-date. I don't run windows so I can't really debug the windows test

@rxbryan
Copy link
Contributor Author

rxbryan commented Feb 21, 2023

I node port it seems there's a bug:

2023-02-21T01:41:33.6835696Z attempt to reference unallocated memory (signal SIGSEGV)

That's not the only problem. There's alot of load_from_file errors... look at this

Traceback (most recent call last):
  File "py_loader_impl_load_from_file_path", line 19, in load_from_path
  File "/usr/lib/python3.9/importlib/util.py", line 90, in find_spec
    fullname = resolve_name(name, package) if name.startswith('.') else name
  File "/usr/lib/python3.9/importlib/util.py", line 32, in resolve_name
    raise ImportError(f'no package specified for {repr(name)} '
ImportError: no package specified for './asd.py' (required for relative module names)
 [Tue Feb 21 00:11:38] #13550 [ 3128 | py_loader_impl_load_from_file_exception | /usr/local/metacall/source/loaders/py_loader/source/py_loader_impl.c ] @Error : Python Error: Exception raised while loading the module './badrequire'
[Tue Feb 21 00:11:38] #13550 [ 4088 | py_loader_impl_error_print | /usr/local/metacall/source/loaders/py_loader/source/py_loader_impl.c ] @Error : Python Error: [Type: <class 'ImportError'>]: no package specified for './badrequire' (required for relative module names)
Traceback (most recent call last):
  File "py_loader_impl_load_from_file_path", line 19, in load_from_path
  File "/usr/lib/python3.9/importlib/util.py", line 90, in find_spec
    fullname = resolve_name(name, package) if name.startswith('.') else name
  File "/usr/lib/python3.9/importlib/util.py", line 32, in resolve_name
    raise ImportError(f'no package specified for {repr(name)} '
ImportError: no package specified for './badrequire' (required for relative module names)
ception in load_from_file Error: Cannot find module './asd.ts'
    at fileResolve (/usr/local/metacall/build/bootstrap.ts:156:25)
    at /usr/local/metacall/build/bootstrap.ts:169:54
    at Array.map (<anonymous>)
    at load_from_file (/usr/local/metacall/build/bootstrap.ts:169:45)
    at /usr/local/metacall/build/bootstrap.ts:44:21
    at metacall_require (/usr/local/metacall/source/ports/node_port/index.js:189:15)
    at Module.mod.require (/usr/local/metacall/source/ports/node_port/index.js:264:11)
    at require (internal/modules/cjs/helpers.js:74:18)
    at /usr/local/metacall/source/ports/node_port/test/index.js:58:26
    at getActual (assert.js:658:5) {
  code: 'MODULE_NOT_FOUND'
}
[Tue Feb 21 00:11:34] #13550 [ 965 | metacall_handle_export | /usr/local/metacall/source/metacall/source/metacall.c ] @Error : Handle (nil) passed to metacall_handle_export is not valid
NodeJS Loader Error: MetaCall could not load from file
Exception in load_from_file Error: Cannot find module './asd.tsx'
    at fileResolve (/usr/local/metacall/build/bootstrap.ts:156:25)
    at /usr/local/metacall/build/bootstrap.ts:169:54
    at Array.map (<anonymous>)
    at load_from_file (/usr/local/metacall/build/bootstrap.ts:169:45)
    at /usr/local/metacall/build/bootstrap.ts:44:21
    at metacall_require (/usr/local/metacall/source/ports/node_port/index.js:189:15)
    at Module.mod.require (/usr/local/metacall/source/ports/node_port/index.js:264:11)
    at require (internal/modules/cjs/helpers.js:74:18)
    at /usr/local/metacall/source/ports/node_port/test/index.js:59:26
    at getActual (assert.js:658:5) {
  code: 'MODULE_NOT_FOUND'
[Tue Feb 21 00:11:38] #13550 [ 1253 | rb_loader_impl_load_from_file | /usr/local/metacall/source/loaders/rb_loader/source/rb_loader_impl.c ] @Error : No module could be loaded
[Tue Feb 21 00:11:38] #13550 [ 965 | metacall_handle_export | /usr/local/metacall/source/metacall/source/metacall.c ] @Error : Handle (nil) passed to metacall_handle_export is not valid
NodeJS Loader Error: MetaCall could not load from file
==13543==T53: FakeStack created: 0x7f67cb5de000 -- 0x7f67cc0e7000 stack_size_log: 20; mmapped 11300K, noreserve=0 
==13543==T53: stack [0x7f67d9f7d000,0x7f67daf7bf00) size 0xffef00; local=0x7f67daf7be1c
CSLoader loading error: File ./badrequire not found
[Tue Feb 21 00:11:38] #13550 [ 965 | metacall_handle_export | /usr/local/metacall/source/metacall/source/metacall.c ] @Error : Handle (nil) passed to metacall_handle_export is not valid
NodeJS Loader Error: MetaCall could not load from file
[Tue Feb 21 00:11:38] #13550 [ 187 | cob_loader_impl_load_from_file | /usr/local/metacall/source/loaders/cob_loader/source/cob_loader_impl.cpp ] @Error : module 'badrequire' not found
[Tue Feb 21 00:11:38] #13550 [ 965 | metacall_handle_export | /usr/local/metacall/source/metacall/source/metacall.c ] @Error : Handle (nil) passed to metacall_handle_export is not valid
NodeJS Loader Error: MetaCall could not load from file

Where should ./asd.py ./asd.ts asd.cs asd.rb be located. I am able to reproduce the load_from_file errors but not the segfault. Maybe they are related.

@viferga
Copy link
Member

viferga commented Feb 21, 2023

I node port it seems there's a bug:
2023-02-21T01:41:33.6835696Z attempt to reference unallocated memory (signal SIGSEGV)

That's not the only problem. There's alot of load_from_file errors... look at this

Traceback (most recent call last):
  File "py_loader_impl_load_from_file_path", line 19, in load_from_path
  File "/usr/lib/python3.9/importlib/util.py", line 90, in find_spec
    fullname = resolve_name(name, package) if name.startswith('.') else name
  File "/usr/lib/python3.9/importlib/util.py", line 32, in resolve_name
    raise ImportError(f'no package specified for {repr(name)} '
ImportError: no package specified for './asd.py' (required for relative module names)
 [Tue Feb 21 00:11:38] #13550 [ 3128 | py_loader_impl_load_from_file_exception | /usr/local/metacall/source/loaders/py_loader/source/py_loader_impl.c ] @Error : Python Error: Exception raised while loading the module './badrequire'
[Tue Feb 21 00:11:38] #13550 [ 4088 | py_loader_impl_error_print | /usr/local/metacall/source/loaders/py_loader/source/py_loader_impl.c ] @Error : Python Error: [Type: <class 'ImportError'>]: no package specified for './badrequire' (required for relative module names)
Traceback (most recent call last):
  File "py_loader_impl_load_from_file_path", line 19, in load_from_path
  File "/usr/lib/python3.9/importlib/util.py", line 90, in find_spec
    fullname = resolve_name(name, package) if name.startswith('.') else name
  File "/usr/lib/python3.9/importlib/util.py", line 32, in resolve_name
    raise ImportError(f'no package specified for {repr(name)} '
ImportError: no package specified for './badrequire' (required for relative module names)
ception in load_from_file Error: Cannot find module './asd.ts'
    at fileResolve (/usr/local/metacall/build/bootstrap.ts:156:25)
    at /usr/local/metacall/build/bootstrap.ts:169:54
    at Array.map (<anonymous>)
    at load_from_file (/usr/local/metacall/build/bootstrap.ts:169:45)
    at /usr/local/metacall/build/bootstrap.ts:44:21
    at metacall_require (/usr/local/metacall/source/ports/node_port/index.js:189:15)
    at Module.mod.require (/usr/local/metacall/source/ports/node_port/index.js:264:11)
    at require (internal/modules/cjs/helpers.js:74:18)
    at /usr/local/metacall/source/ports/node_port/test/index.js:58:26
    at getActual (assert.js:658:5) {
  code: 'MODULE_NOT_FOUND'
}
[Tue Feb 21 00:11:34] #13550 [ 965 | metacall_handle_export | /usr/local/metacall/source/metacall/source/metacall.c ] @Error : Handle (nil) passed to metacall_handle_export is not valid
NodeJS Loader Error: MetaCall could not load from file
Exception in load_from_file Error: Cannot find module './asd.tsx'
    at fileResolve (/usr/local/metacall/build/bootstrap.ts:156:25)
    at /usr/local/metacall/build/bootstrap.ts:169:54
    at Array.map (<anonymous>)
    at load_from_file (/usr/local/metacall/build/bootstrap.ts:169:45)
    at /usr/local/metacall/build/bootstrap.ts:44:21
    at metacall_require (/usr/local/metacall/source/ports/node_port/index.js:189:15)
    at Module.mod.require (/usr/local/metacall/source/ports/node_port/index.js:264:11)
    at require (internal/modules/cjs/helpers.js:74:18)
    at /usr/local/metacall/source/ports/node_port/test/index.js:59:26
    at getActual (assert.js:658:5) {
  code: 'MODULE_NOT_FOUND'
[Tue Feb 21 00:11:38] #13550 [ 1253 | rb_loader_impl_load_from_file | /usr/local/metacall/source/loaders/rb_loader/source/rb_loader_impl.c ] @Error : No module could be loaded
[Tue Feb 21 00:11:38] #13550 [ 965 | metacall_handle_export | /usr/local/metacall/source/metacall/source/metacall.c ] @Error : Handle (nil) passed to metacall_handle_export is not valid
NodeJS Loader Error: MetaCall could not load from file
==13543==T53: FakeStack created: 0x7f67cb5de000 -- 0x7f67cc0e7000 stack_size_log: 20; mmapped 11300K, noreserve=0 
==13543==T53: stack [0x7f67d9f7d000,0x7f67daf7bf00) size 0xffef00; local=0x7f67daf7be1c
CSLoader loading error: File ./badrequire not found
[Tue Feb 21 00:11:38] #13550 [ 965 | metacall_handle_export | /usr/local/metacall/source/metacall/source/metacall.c ] @Error : Handle (nil) passed to metacall_handle_export is not valid
NodeJS Loader Error: MetaCall could not load from file
[Tue Feb 21 00:11:38] #13550 [ 187 | cob_loader_impl_load_from_file | /usr/local/metacall/source/loaders/cob_loader/source/cob_loader_impl.cpp ] @Error : module 'badrequire' not found
[Tue Feb 21 00:11:38] #13550 [ 965 | metacall_handle_export | /usr/local/metacall/source/metacall/source/metacall.c ] @Error : Handle (nil) passed to metacall_handle_export is not valid
NodeJS Loader Error: MetaCall could not load from file

Where should ./asd.py ./asd.ts asd.cs asd.rb be located. I am able to reproduce the load_from_file errors but not the segfault. Maybe they are related.

Those errors are fine. I am reviewing everything, you overwrote all my changes. I am going to try to solve them.

@viferga
Copy link
Member

viferga commented Feb 21, 2023

I think I almost get it to work, I want still to fully review it locally.

@viferga
Copy link
Member

viferga commented Feb 22, 2023

Yeah, it seems all tests passed, but I still want to fully review it. Formatting failed because I modified it from the web, but I also want to fully review all the implementation properly, I've seen things that are not clear to me and I need to understand them better and provide better feedback to you. Good job!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants