Architecture design #1
Comments
As long as this feature is by default, and not mandatory, I'm fine with that. I just don't want the GUI builder to restrict functionality, i.e. the user should easily be able to adjust |
Yes, I agree. It should be possible to manually modify the configuration and generated C code. One complication is: do these manual modifications need to propagate to the editor when a project is reopened? Can a project be imported from an exported code? If so, how? This may help argue the case for an intermediate (ie: XML) representation of the GUI. |
If you are using micropython, it is easy to know the configuration that was set in lv_conf.h. For example, if you want to know if some object was enabled, just check if there is such class in lvgl module. Python introspection is handy in such situations. |
|
I have worked with pytkinter and wxPython before and haven't been very impressed with either. I asked a colleague for recommendations for a GUI library with a Python binding and he suggested pyGObject/Gtk. It has a graphical GUI builder (Glade). I haven't used it before but I'm going to play around with it a bit. It is cross-platform as well. Does anyone have a preference for the GUI library? |
|
@AGlass0fMilk I think @amirgon wants to use LittlevGL as the GUI for the GUI builder. |
From a performance standpoint that doesn't really make sense. The It would be a cool proof-of-concept or showcase project but I think it wouldn't perform very well as a tool. |
I don't agree. There is no reason why you wouldn't get decent performance with lvgl as a GUI. If you are missing some "more complex built-in controls" then add them to lvgl and it would be a useful addition to lvgl in general. It makes more sense to use lvgl as GUI since you want to display and manipulate lvgl components, and lvgl already contains all the logic of how to draw and manipulate them. |
|
To decide it let's collect which controls are required to build up the GUI designer. I have these in mind:
Complex ones:
It really would be great to create the GUI builder with LittlevGL because:
However, in this case, I think we should use the Python binding of LittlevGL (instead of Micropython) to enable to use of Python libraries. |
@kisvegabor Which libraries did you have in mind? |
|
@kisvegabor Is the Python binding complete enough to be usable within our timeframe? Ideally we could release the GUI builder alongside |
|
My suggestion is that we build a GUI builder tool targeted towards efficient desktop computer use. Then, using the tool, we can build very complex, beautiful UI that is easy and efficient to create/design. The UI we create with the tool can showcase the capabilities of LittlevGL. Using a desktop-oriented GUI library to build this tool makes sense for a few reasons:
I just don't think LittlevGL is the right tool for desktop application development. It will just take longer to develop, run slower, and be more frustrating to use. We can probably make a GUI builder with a desktop GUI framework and use the time we save to create some of the more complex widgets before V6.0 is released. I have been playing with Gtk3 and Glade, I should have a "LittlevGL widget" working in it today or tomorrow. |
|
I agree with @AGlass0fMilk. It's probably simpler to make the GUI with a proven desktop tool. |
|
Personally, I liked the idea of lvgl-GUI-Builder-built-in-lvgl better. In my opinion it has a larger potential to drive lvgl forward.
I'm not sure any of these claims is true. lvgl is easy to develop with, can run fast enough and any time spent for adding a missing feature is a pure gain since it will help taking lvgl library forward. One more thing. If we want to develop a modern application, why not a web application? Think of an online tool available on lvgl website that allows the users, with their web browser, design and run lvgl GUI! No need to install anything, no need for a precompiled version for each OS, etc. We are half way there with lvgl/lvgl#687 and lvgl/lvgl#792. There might also be other ways to achieve this, by using JavaScript and some web framework. |
|
@amirgon Please explain why you think using LittlevGL for the graphics library would be better.
It brings a lot of improvement. With this the user no longer needs to spend time fiddling with the coordinates and sizes of objects on a display. Now they can just set everything up on their screen visually.
@AGlass0fMilk already highlighted some problems which make sense:
|
How is it different from lvgl-GUI-Builder-built-in-lvgl? It will also let the users just set everything up on their screen visually. I just don't get your point.
But our target audience is not desktop users, it's embedded device users. Having GUI components that are not mocked but the real thing, look and behaves like the real thing, seems more important to me.
lvgl already runs on many OSes and platforms, so lvgl-GUI-Builder-built-in-lvgl will too. This is also already taken care of.
I agree that there are some complex GUI components that exist in desktop application and are not needed on lvgl. But let's try to list them. For each one, we need to ask ourselves:
Let's see how many such relevant complex component we find. |
This feature/project pertains to a GUI builder for the lvgl library, not the lvgl library itself. You can still contribute to the lvgl core library in parallel.
We are targeting embedded developers so I don't see much advantage to a web-based application. A desktop-based application will have more support for additional features (such as interacting with and uploading to attached target hardware). I think it would be more work to write a GUI builder application using a JavaScript wrapper for a Python wrapper for a C library. Mbed-OS has an online IDE and compiler toolchain. It is great for hobbyists and beginners but I never use it. I have only ever been limited by the online IDE. Embedded developers typically require finer control over their development toolchain and as such, desktop/CLI tools are pretty desirable. We could even provide command line utilities for integration into CI/CD pipelines if the need arises. This would be more difficult with a web application. Also, a web application would be much harder to add custom (non-public) widgets to as well. Additionally, we will have to host the web application, possibly provide online storage for users, perhaps authentication... complexities that don't have any measurable benefit for professional embedded developers.
This isn't very relevant. One will always have to be learning frameworks and libraries to stay current in this field.
Actually this isn't true. My idea is to use a generic canvas for displaying the LittlevGL rendering. We can port LittlevGL to write its display buffer out to this canvas. This will eliminate any inconsistencies in appearance (barring any bugs in the wrapper layers).
The GUI builder will need to be aware of lvgl widget object properties in order to configure them. Thankfully, python introspection should take care of most of this, allowing us to enumerate properties and create the appropriate controls in a side-pane based on the property type. As for the specific alignment issue -- there are only a few alignment types in lvgl that can be specified. The user can simply choose an alignment type from a drop-down list and see if it has the desired effect in the live-view GUI canvas.
This feature/project pertains to a GUI builder for the lvgl library, not the lvgl library itself. You can still contribute to the lvgl core library in parallel. Web-applications and desktop-grade widgets seem like scope creep to me.
Embedded developers use desktop applications for development. We can have the actual LittlevGL library render the live-view while using native UI for manipulation and control so that point is moot.
Cross-platform GUI libraries/frameworks are plentiful, so this is a non-issue.
They would be familiar, intuitive additions to the GUI builder's interface that would help most developers learn how to use stuff without even reading documentation. These are also widgets/features that are not really needed by most applications of LittlevGL. Yes the mouse cursor is laggy
I have already mentioned a number of features that would be missing if we developed the lvgl builder with lvgl. See comments above. |
|
I fully agree with all of @AGlass0fMilk's comments. I see no reason to add additional work by making the GUI builder with LittlevGL. |
|
I try to group the mentioned questions: Use LittlevGL to create the builder itself or not?
My conclusion It'd be interesting to use LittlevGL this way but I see more benefits of using an existing desktop GUI library. Online or offline GUI builder?
My conclusion I see more advantage in the offline editor Python or Micropython binding?
My conclusion Still there are open questions but if speaking about an offline desktop tool written in Python, the Python binding seems more rational. |
|
The examples from pylvgl use pyQT5 as the backend. I used a similar process to create a GTK-based LVGL display. See screenshot below (just demo UI, not any kind of mockup). It's nice because GTK automatically scales the LittlevGL canvas up on my HiDPI 4k display. QT displays it at 1:1 resolution and it's kind of small and hard to see on my display. Though it does look a bit pixelated, there are options to change the scaling factor.
The Python binding is nice, I'm a little weary of the "missing features" you mention @kisvegabor. I'll see what I can get working with the current binding. |
It has been a while, but I can start again spending some time on this. Now that there is a dev-6.0 branch, I can use that to integrate Python & MicroPython bindings, for which I already did quite some work a while ago. I can assist in focussing on specific things if they are required for the gui development.
@AGlass0fMilk if you have any specific requirements, let me know and I can focus on them. |
|
Regarding the architecture: I'd agree with @kisvegabor 's conclusions. I have been thinking about a GUI builder for a while (even before this discussion) and I would think of an offline Qt (or similar) Python application, which uses the lvgl Python bindings to draw the gui that is being designed on a canvas. The result can be exported as C, Python, or XML. The user can interact with the canvas to move and select objects. There would be a property window and a parent/child tree that represent the current structure of the objects in LittlevGL and allow them to be modified, like the Qt Designer:
The properties that each object has, and their datatypes, can easily be inferred using the Python introspection features, as @amirgon suggested. |
|
@rreilink Yes, that's what I was thinking. My first thought is to use an XML internal representation. Then, we can develop converters that take the XML and generate the appropriate code for whatever language is being used. GUI projects can then be saved in the XML format and imported. This wouldn't allow for code changes to be back propagated to the GUI design however. The GUI designer could internally use XML and a corresponding Python converter to generate the live view. Probably should only process objects that change to prevent having to regenerate the entire live view code in real time. I'm not 100% familiar with how things like argument lists and types propagate from C to a Python binding. It seems like not all the information is being supplied to the Python binding when I inspect members of the For example, to build the When I look at I have seen other Python bindings that have more information, like documentation in the help pages and argument lists that have more detail than "...". @rreilink Do you know how we could have this kind of information propagate from the C code to the Python binding? I would also like to possibly have additional metadata from specially formatted comments (similar to doxygen). For example, Glade, the GTK editor, has a nice feature where tooltips pop up over widget properties to explain what that properties is/does. This would prevent a lot of instances where you have to look at the documentation for something that isn't completely clear. See example of this in screen capture below:
See #3 for that discussion. |
|
@rreilink I suggest opening an issue in @AGlass0fMilk Let's discuss the details of tooltips and helps in #3 Having a layout design of the UI ( just a simple scratch) to see how the control elements are organized would be important already in this phase too. |
I just did :-) |
|
Began work on PyQt-based GUI builder. Being somewhat blocked by the state of the pylvgl binding. See the initial-development branch.
|
|
Awesome! Looks good. It seems it has everything we have discussed. |
|
I have a basic lvgl simulator working with the newest release of pylvgl. I decided to use Currently, the lvgl framebuffer is drawn as the background layer. This will allow us to overlay builder-related UI on the lvgl simulation (such as highlights on selected items, etc).
|
|
Excellent! This is looking great thus far. |
|
Wonderful! I'm very happy to see it working. :) |
Instead of dragging the element itself, how about painting the element on a canvas and dragging only the element's "image"? When the user places the element, you can create it there as a real element. |
If you are referring to dragging the canvas object, that could work, but it would add additional complexity. I think the method that @kaiakz is using now is fine. |
|
This gif shows where my implementation currently stands as a desktop/Qt-based application. All the editable parameters and placeable objects are inferred using python introspection so effort to support littlevgl updates and new widgets is minimized:
Still need to implement dragging and resizing objects using the mouse. Then saving and exporting to code. |
|
@AGlass0fMilk Congratulations! |
With multi-display support, it's possible to create a dummy screen whose buffer is the canvas thus after a refresh LittlevGL will draw objects there. If you are interested I can shaw an example. @kaiakz |
|
@kisvegabor @embeddedt @amirgon
I am interested in it, for I want to support more feature. |
|
@kaiakz Nice! Instead of making the user download multiple files immediately, maybe it's better to let them preview the header and source files, and then choose to download them (or copy them). You might want to indent child objects based on their position in the hierarchy. I was confused about why all my objects were being created as children of the button - until I realized that objects are created as children of whatever you have selected. |
I think the title
OK, I will add it to TODO. I think I can use monaco-editor to support the preview. |
|
@kaiakz I agree with @embeddedt about indentation of child objects. Maybe the attributes can be grouped somehow too. (obj, or text area attributes) |
@kaiakz The demo doesn't work for me, tried both Chrome and FireFox. It just seems loading the page indefinitely. Am I missing something? |
I am sorry for this. But I haven't changed the demo yet. |
|
Working for me again now (online version). It was probably just an issue with GitHub Pages. |
|
@kaiakz It's working for me now. Very nice. What would be the next step? What are your plans further?
|
I think lv_mpy can add more information to JS port, not only parametres (how many? what) that store in Python tuple/list, but also more LittlevGL doc(It also could provide by JS parts). Luckily, Micropython has ujson for converting.
Like above, store them first. For example,
A little, because a widget has two ways for alignment: X-Y position(drag&drop to decide), align to another widget. I will open an issue to discuss the alignment and layout.
I develop a
Sure! Since walv always generate the mpy code at first, and then convert the code to C. It's not difficult to save mpy code. By the way, walv chooses the lv_mpy REPL as its console that provides another way to control walv. Use the walv, user will get in touch with lv_mpy. |
Basically all public API functions are necessary but it's not trivial how to order them. In the C header files, I tried to order them as most important first but I don't know if this order is preserved in MicroPython. |
|
We could have a JSON file that stores the categories of properties, and if an unknown property is found, it just gets put at the bottom of the list. But I wouldn't worry about order right now. Let's get the basic functionality working first. |
I can create a JSON "metadata" file that gives more information about lvgl objects, functions, parameters, enums etc. You could, if you want, write some text processing script to extract comments from the code and put them in some JSON file, but that would be out of the scope of the lvgl binding script. |
|
The Micropython binding can now generate metadata, to allow full introspection. Here is an example: https://raw.githubusercontent.com/littlevgl/lv_binding_micropython/master/gen/lv_mpy_example.json I've enabled it on lv_micropython, so it's now generated automatically into the "build" directory of the port you are building (look for I didn't include metadata generation for structs because I think structs can be introspected in Micropython without the need for metadata (just recursively |
|
@amirgon I had an idea for another LittlevGL project where I would want metadata like this, but not for use with Python. How hard would it be for me to modify your script to generate metadata for the C functions instead of Python functions? I should only need parameter types and return values. |
@embeddedt The metadata generation is fed from internal data structures on the Micropython binding script, so I don't think that by itself it could help you for C. But I suggest you use another approach: use pycparser directly. |
|
@amirgon @embeddedt MicroPython v1.9.4-1626-g2940838bf-dirty on 2019-09-15; JS with Emscripten
Type "help()" for more information.
>>> import lvgl as lv
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: no module named 'lvgl'Maybe the emsdk did not compile and link the something about |
|
@kaiakz - You are probably using the "master" branch and not the "lvgl_javascript" branch. This is a recent change. Until recently the master branch included lvgl in the js port, but now we removed it and added lvgl only on "lvgl_javascript" branch - so you must switch to that branch. |
|
@embeddedt @amirgon Use make V=1 or set BUILD_VERBOSE in your environment to increase build verbosity.
GEN build/frozen.c
GEN build/genhdr/qstr.i.last
cc1: error: argument to ‘-O’ should be a non-negative integer, ‘g’, ‘s’ or ‘fast’
cc1: error: argument to ‘-O’ should be a non-negative integer, ‘g’, ‘s’ or ‘fast’
...
cc1: error: argument to ‘-O’ should be a non-negative integer, ‘g’, ‘s’ or ‘fast’
make: *** [../../py/mkrules.mk:75: build/genhdr/qstr.i.last] Error 1
make: *** Deleting file 'build/genhdr/qstr.i.last'My env is : Set the following tools as active:
releases-fastcomp-3b8cff670e9233a6623563add831647e8689a86b-64bit
node-12.9.1-64bit
emscripten-tag-1.38.25-64bitWhen I installed emsdk install clang-tag-e1.38.31-64bit
Installing tool 'clang-tag-e1.38.31-64bit'..
The contents of file 'https://github.com/kripken/emscripten-fastcomp/archive/1.38.31.tar.gz' already exist in destination '/home/kai/Templates/emsdk/clang/tag-e1.38.31/src', skipping.
The contents of file 'https://github.com/kripken/emscripten-fastcomp-clang/archive/1.38.31.tar.gz' already exist in destination '/home/kai/Templates/emsdk/clang/tag-e1.38.31/src/tools/clang', skipping.
Running CMake: ['cmake', '-G', 'Unix Makefiles', '-DCMAKE_BUILD_TYPE=Release', '-DPYTHON_EXECUTABLE=/usr/bin/python3', '-DLLVM_TARGETS_TO_BUILD=X86;JSBackend', '-DLLVM_INCLUDE_EXAMPLES=OFF', '-DCLANG_INCLUDE_EXAMPLES=OFF', '-DLLVM_INCLUDE_TESTS=OFF', '-DCLANG_INCLUDE_TESTS=OFF', '-DLLVM_ENABLE_ASSERTIONS=OFF', '/home/kaiakz/Templates/emsdk/clang/tag-e1.38.31/src']
Traceback (most recent call last):
File "/home/kaiakz/Templates/emsdk/emsdk.py", line 2891, in <module>
sys.exit(main())
File "/home/kaiakz/Templates/emsdk/emsdk.py", line 2871, in main
success = tool.install()
File "/home/kaiakz/Templates/emsdk/emsdk.py", line 1659, in install
success = build_llvm_tool(self)
File "/home/kaiakz/Templates/emsdk/emsdk.py", line 1081, in build_llvm_tool
success = cmake_configure(cmake_generator, build_root, fastcomp_src_root, build_type, args)
File "/home/kaiakz/Templates/emsdk/emsdk.py", line 979, in cmake_configure
ret = subprocess.check_call(cmdline, cwd=build_root, env=build_env(CMAKE_GENERATOR))
File "/usr/lib/python3.7/subprocess.py", line 342, in check_call
retcode = call(*popenargs, **kwargs)
File "/usr/lib/python3.7/subprocess.py", line 323, in call
with Popen(*popenargs, **kwargs) as p:
File "/usr/lib/python3.7/subprocess.py", line 775, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.7/subprocess.py", line 1522, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
PermissionError: [Errno 13] Permission denied: 'cmake'But make Set the following tools as active:
clang-tag-e1.38.31-64bit
node-12.9.1-64bit
emscripten-tag-1.38.25-64bit |
|
You installed a different version of clang than Emscripten. |
Thanks! All problem solve after I reboot my computer, and I finally got the completed js files with the environment above (though different version clang). The micropython.js is large and sometimes slow(than before), maybe it needs optimizing. |
In continue of lvgl/lvgl#601
So according to my current understanding, the concept is the following:
I expect that some extensions should be developed. For example one basic thing came to mind: when you click on the canvas to select an object to edit it, the object on that coordinate should be get and highlighted.
The text was updated successfully, but these errors were encountered: