-
Notifications
You must be signed in to change notification settings - Fork 647
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
On-chain user scripts ("dumb" contracts) #1182
Comments
Nice ! |
Maybe we should distinguish
A stateless contract just consists of an instruction set with certain inputs. Think sharedrop or dividends. Those are automated transfers to multiple/all accounts. No state is required, all incoming tokens are distributed. Another example is Bitcoin's Script. A stateful contract, in contrast, is much more difficult I think. Mostly because the block producers would need to validate state consistently as well. I would argue that this is out of scope for BitShares. |
NOTE INFO
|
I did some brief research a while ago looking into the possibilities of supporting some scripting language inside the core. I think the solution could be done in the form of a plugin and do not modify the core logic at all unless really needed. Plugin should expose an API to upload script file and plugin will process it. On my research i found that LUA plays nice with c++ and it seems easy to get started and build some sort of prototype. LUA is MIT(https://www.lua.org/license.html), LUA was(maybe still is) used in some well known trading platforms like Trade Station among other reasons i don't remember now, LUA was the best candidate of that moment IMHO. I can do some more research on this now that it is posted here. Whatever path we follow, effort to make this well will be huge as a lot of details need to be considered but as an optional plugin it haves more possibilities to be done and adopted and scaled. |
In regards to @cogutvalera 's comment. HyperExchange uses LUA and GXB uses WASM. I have no experience when it comes to scripting language integration so I can't comment on specifics. I assume LUA (with its huge ecosystem) would be easiest. My concerns are regarding performance which is why I was suggesting a minimal solution by keeping the featureset small. Also, considering this suggestion is about ON-chain scripts, which would need to be ran deterministically when the "account" object holding them is impacted and consensus reached, how is it possible to run this as a plugin? |
absolutely agree with @clockworkgr about performance concerns, but also I've concerns about security issues for such smart contract feature |
I was thinking on something here, correct me if i am wrong. If we want to be adding scripting i suppose one of the goals will be to be able to make trading strategies with some logic in the node with this capability enabled. So, a program can at most check object state from a script instead of using an API but to change the database it needs to execute an operation and follow the normal path. Will this bring real benefit ? I am trying to find the use cases mostly from trader perspective before further research. |
@oxarbitrage basically you're talking about bot logic but not on-chain logic. "Sending private key" is actually done by signing a transaction. |
Just to clarify, My initial suggestion was to treat those "contract"/script accounts as special and have them produce ops that are included(or at least derived) but need not be signed in a way. similar to virtual ops. For example, If I send 10 BTS to my "dividend" script, the chain would run through the "script" and end up creating 2x 5BTS transfers out to two different accounts. Those 2 transfer ops, seeing as the script's balance is enough and can not be created any other way then via the chain interpreting the script in a deterministic way, would not have to be signed. Only my initial transfer of 10 BTS would have to be checked as usual. The rest is derived by that tx combined with the (deterministic) script execution. Does that make sense? To be honest I would like to see some degree of statefulness to the scripts. However I can see that for example historical lookups would not be possible (witness nodes carry no history) but certain actions could be taken based on current db state and perhaps keep a "contract" state (of limited size) somehow. |
Thinking about it further, I think my initial suggestion to treat them as accounts is probably best. Essentially, it's a "special" account that can do anything a normal account can do except it has no keys and the ops it performs need not be signed as long as they are ops that would need its own authority. For example, assume my script lives at 1.2.3456789. Any op the script produces that would require the authority of 1.2.3456789 is considered signed seeing as there is no other way for those ops to be entered into the chain then via the script output. That's the output side done. On the other side we need to figure out the input side and when the script needs to be ran and evaluated. Here we have 2 cases. a) Either the script account is impacted by another chain operation (say someone sends funds to 1.2.3456789) in which case we run the script via the appropriate handler. We could also have the script "list" its used handlers on creation so we can quickly ignore ops that it's not supposed to handle without interpreting it for better performance. b) Or a specific method in the script is called (this would require a new op_type call_script_method).... This would have to be signed by the script "owner" for security reasons. We could always have the script check the method caller (similar to what ethereum does) but that would incur a performance hit as the script would have to be interpreted first. Putting this altogether: I , clockwork, (1.2.711128) create a new 'contract'/'script' at 1.2.999999. I am now the owner of that script. The script creation operation also takes a list of 'impacts' this script is equipped to handle. For simplicity's sake, let's say I list only 'BalanceChanged'. This means that the script will be interpreted and executed via the onBalanceChanged handler whenever an operation (any operation) changes the script account's balances. So for example if a 3rd party sends it an asset...or a gateway issues an asset to it. Any resulting operations from the script's execution would be handled like virtual ops, deterministically changing the db state as long as it's something the script is allowed to do (like for example transferring part of its own balances out). Let's also assume that I have a 'public" method in the script. Staying with the dividend simplistic example, we call it setMinimumBalanceForDividends(). This method can be called via a new op_type The script would be interpreted and executed IFF the call_script_method was signed by the script account owner. Or , assuming custom authority BSIP gets implemented by someone listed in the custom authority for that op_type. In that case , the original owner could set asserts for the script_account and the method_name thus allowing me to do stuff like: Active authority is K if script_account = 1.2.99999 and method_name = xxxxxx Thus making different parts of the script "controllable" by different entities. Moving onwards. Let's assume the script can store some state data (extension of account statistics object perhaps?) then the setMinimumBalanceForDividends updates a state variable store with the new minimum balance. Going back to my original pseudocode example:
Let's call this script Account The new 'special' account Now account Using custom_authority (if implemented) and the required asserts he could also delegate access to these methods to one or more other accounts as required. Whenever the script account's balance is changed, the script is interpreted and the onBalanceChanged method executed. If impacted by any other op, it is ignored (since we have not registered any other handlers on creation). If any of the methods (in our case, onBalanceChanged) result in new operations being created/db state being changed, we update state accordingly essentially treating the output of the script as 'signed'. Obviously it can only do the ops its allowed but at least we shouldn't have to worry about sig verification. I'll stop here but please consider also the case where the script account itself is added as a custom authority to another (non-script) account. We could have very flexible scenarios this way. |
Could a script perform an ES lookup, for say distributing an asset's collected market fees to users who traded the most (for a single trading pair) during a set period of time? The balance redistribution idea is good, but could you provide weights for different users (ie user A gets 1, user B gets 2) based on a Bitshares blockchain query? How would cost of computation and flexible target user size factor in? Would a simple script that splits income between 2 users cost the same as someone targeting the whole BTS holder's list for a regular UIA issuance distribution? |
Balance redistribution was just an example. Consensus using external data would be impossible (unless some kind of oracle setup was added) and ES data is external. The data that a script could access , would be whatever is in the consensus DB state. this excludes account history/market history etc. I guess we could move some potentially interesting useful data to statistics objects but I believe we already have a decent base to cover quite a few use cases as it is. As far as costs/fees are concerned, I haven't got that far yet. |
OK, so if we can get targets but we cannot assign weight due to requiring external factors, could a group of (asset owner defined) users provide weights against the retrieved targets with a new op? This way price feed publishers could be performing complex external computations then providing a list of weights which the script would take the median of to provide the weight-applied asset-issuance | fee-redistribution? |
Using witnesses for Oracle type work is what we already do for pricefeeds... I guess we could do the same |
@clockworkgr , @ryanRfox , what is the status of this issue? Is there a BSIP for it? We in OpenLedger have played with Ethereum VM and managed to integrate it with Bitshares core, so I wonder if this is something that may be interesting to the community. Please advise) |
@charyury I've been slowly piecing together the steps that need to be taken for this to work properly in discussions with @oxarbitrage . In fact, yesterday we were discussing how our biggest issue is finding the right VM/scripting interpreter that will allow us to calculate script usage/metrics so we can set fees appropriately. Seeing as EVM is one of the few VMs that already implement this I was toying around with taht idea myself (although I'm worried about its performance) Is there somewhere I can read more about or see your implementation? |
I had been testing in the last few days 2 different languages approach that are lua and wren. they both have active and similar binding libraries: My original plan was not the same as what is discussed here but related and at the end they should be joined. I have in mind a plugin that will accept scripts(lua and wren are where i have some code done) that will be able to get some data from the blockchain ( To make it the private key will need to be present in the script or similar(insecure). Custom authorities can bring another level of security to the script by using the trading auth. At the end the client will have to trust the node where script are being executed, can be his own node with the Then, to make it in the consensus level(signatures will not be needed) the witness need to execute loaded scripts on each block, get the same results and execute trades without the need of signing. @clockworkgr haves a process for it but i was not able to fully understand it, the hardfork part should be small and easy. Need to formalize. @charyury will be great to hear more about your testing and how that part was solved specifically, maybe we can work together if we want to change the vm, i am not sure if ethereum will be ideal. |
Here is a short instructions how to reach the testnet and run a smart contract. You may load any Solidity contract from Ethereum. You may also operate Bitshares objects. Enhanced CLI also provided. Will post more documentation later. |
After numerous discussions with @oxarbitrage, @Zapata et al, I'll try and go back to basics and spec out/explain a possible implementation strategy trying to piece together the previous conversations. WHAT WE WANT Essentially we want the script to function as a normal user account. Just like I as a person would monitor stuff on the chain, react to a state change on the chain, perform some logic in my head and then possibly broascast some transactions in reaction to that, we want the contract to do the same. The only difference is that instead of a person thinking, it's the script being evaluated and deterministically reaching a result in an expected and trustless way. With that in mind, script/contract objects SHOULD be the same as account objects, they have the exact same capabilities/feature/data + a few extra things. We can either use a new space for them such as 1.16.XXX which seems to be the way OL implemented it, but that would lead to duplicate code and extra changes in clients. Hence why my intital suggestion was for scripts/contracts to live as normal account objects (1.2.XXX). NEXT STEPS The following is a suggested path for implementation that borrows heavily from ethereum concepts. As a first step, we want a way to "create" and "store" those script accounts. Storing is 'easy' as we can simply add additional data fields to the relevant script-account object or its associated statistics object. When creating however, we have an added requirement. We need to store the "owning" account. By default, this is the only account/authority that can call methods on the script. Again, this could also be an additional field. Therefore, let's assume the new operation: If my account (clockwork) signs and broadcasts c
Now, going back to the Ethereum metaphor, ETH only evaluates the contract if a) a method is called or b) a transfer to the contract address is made. In our case we have more possibilities to identify WHEN to evaluate a script. First of all, a script method being called. For this, we create a second op: By default, we have this locked down with its evaluator require active permissions of the script owner (as defined in the field mentioned previously). So only the script creator/owner can call a script method. Assuming implementation of BSIP40, we can open up those permissions as needed via custom_authorities. Our second case is evaluating the script if it's state has been changed. In Ethereum we only have a transfer (i.e. balance change) but in BitShares, an account may be impacted by other ops too. The good thing is we already monitor for those in -core to handle object subscriptions. In order to avoid unnecessarily evaluating a script or running unnecessary code, we should force scripts/contracts to declare their USED handlers on creation. So going back, our create_script_account_operation now has a signature of This concludes our first step. We now have a script on-chain, with a defined owner and defined entry points for evaluation (either via method call or via other chain op impact) that has all the properties and capabilities of an account but cannot perform ANY ops through tx broadacsting due to lack of keys. The second step is evaluating the script/contract while retaining consensus. For this, and in order to limit the impact on the existing consensus code, I propose that block production is split into 3 stages. First, we evaluate and apply all existing ops as we do right now. If any of them 'impact' an account object that is flagged as script and THAT script-account lists that impact-type as one that it handles, we put an 'impact-type','script_id' data-pair into a queue for processing. Once this is completed, we take all our call_script_method_ops to be included in that block (if any) and if they are signed properly (by the script owner, or an authority the owner has delegated to via BSIP40) we evaluate the script accordingly via the relevant method entry point. Finally, we take our queue from the first step and evaluate the scripts using the relevant handlers as entry points. Same input data -> Same scripts -> Same order => Same Output and thus consensus. Now, the final step is that in the process of evaluating the script, the script is freely capable of producing ANY bitshares operation as a "virtual" operation. It being a product of valid inputs and consensus, we don't require keys for it in the sense that we assume the script-account's active authority as a given. So the script can create ANY operation that would normally require the script-account's active or owner authority. We are thus able to use the existing evaluation code for the validity of those ops with hardly any changes. ECONOMICS & DoS PROTECTION Whatever VM/script interpreter we choose, should provide metrics (similar to the EVM) and opcode count. Just like ethereum, we should extend the fee-schedule to have an opcode/fee table and calculate fees used as the script is evaluated. If the script opcode count exceeds fees paid by the relevant method call (or in remaining balance of the contract for impact handlers) we stop execution and no state change has taken place. Otherwise, the evaluation completes successfully. This means that the relevant UI should be able to evaluate the script off-chain and suggest the appropriate fee (UI pulls script state, script code, suggested TX and calculates the resulting opcode count & associated fee) We should also set limits on the total number of opcode calls and script state storage. I also suggest that a percentage of script-related fees is paid directly to witnesses. |
@clockworkgr , it seems that starting from some kind of simple scripting solution, the issue is moving towards powerful and protected business logic processing mechanism, executed on consensus level, which is very close to our solution. I think the best way to proceed is describe our solution in details and see how it is compared to the requirements proposed by you. This is in progress already, I expect to share the information by the end of the week. |
i also have some work in progress with a plugin that i will share more in details as soon as i have something decent to present as a proof of concept. |
Then why not implement it client-side? IMO before speccing out a major change like this we should answer the question - do we want on-chain scripting AT ALL? |
@pmconrad client-side is essentially what @oxarbitrage 's solution is tackling. Whether we want on-chain scripting is the big question. Personally I would as I see many advantages , as long as it doesn't come with a huge performance hit. The discussion is not so much about speccing it out but (at least on my side) to simply explore possible implementation paths in order to better evaluate if it's worth it. Hence my request for comments was simply to see if this proposed architecture even makes sense at all. |
my approach is client side with the help of a bitshares plugin that will execute client scripts in a VM created by the plugin on every block. |
Our approach is all about on-chain. We believe smart contract is something that modern blockchain must have. Bitshares is perfect blockchain for business solutions, smart contracts is just one last thing still missing. Regarding performance, it is more a matter of correct pricing model. More paid transactions means more activity in the ecosystem, this is what all stakeholders are looking for. At this time Bitshares is definitely underutilized, there is huge space for transactions count increase. |
What is the difference to having the owner permission of the new script account to be the creating account?
I suggest not allowing anything that has to do with owner authority.
Witnesses have their pay already, what is reasoning here?
For my clarification:
My 2 cents on economics: |
More details about our solution in the attachment. Looking forward to the core team members feedback! |
@charyury i think the document looks great. I had to change my solution from pure plugin to execute the scripts in the chain as i found no way to make it secure without consensus. The solution is similar to the one you are providing in the high level, probably different internally as it use wren as vm instead of ethereum. solution could be adapted also to lua and luajit if wren is not good enough. It was not my intention to make the 2 solutions to compete, if i knew openledger was working on something similar before starting i will be probably on something else. The problem i see is that having both vm in the consensus running will be unlikely. Opinions welcome. Should we work together ? |
@oxarbitrage, how far are you with you solution? Do you have something tangible already? As you can see, we have quite a big progress, so it is a good time to test our solution and decide if this approach fits the network. If you are somehow close to the same level of readiness, probably we shall compare both solutions in more details to decide which one is better and then go for it. Speaking frankly, we are not much aware of wren and lua, so we might miss some big advantages those have. Could you share your vision on pros and contras? In our opinion Ethereum VM has two big advantages: it is well known and has big developers community behind it, and there are tons of smart contracts already implemented that can be ported to Bitshares w/o any updates, so any business solution that uses Ethereum at this time can migrate to cheaper and faster environment immediately, and benefit from using DEX and MPAs that Bitshares has at the same time. Anyhow, we would be happy to collaborate! |
The process is similar. Users upload scripts to the chain with operation. On each block bitshares-core will execute each active script in a wren VM and do the needed operations. It can currently only do transfers but can do other core operations. It haves similar functions to get data from the network, a simple script may look:
or
etc. It is not developed as much as your pdf but it is most likely going after a very similar output. As a pro of your system i agree with the 2 main advantages of ethereum however not sure if the community/stakeholders will agree on that. In my opinion i don't see anything wrong with that if it works for our needs. In regards to how far, i have some code already that i will share in the next few days as i need to do some cleanup and changes. I cant leave it incomplete at this point but i will not advance much more after that unless the solution is considered. That will be good enough as a proof of concept. We can make some comparisons then, an alternative. It will be a good idea to make tests on both and see what we get from there. I will not try to stop you to get my idea on front, in the contrary, i respect the work you had done until now, will help in what i can. Also mine is actually just 1 week of work at this moment. |
@oxarbitrage how do you imagine solve the economic part with wren/lua solution? |
You cant know upfront how many transfers a contract will end up executing as it will depend on conditions, can be 1 , 100 or none. For usage, as a simple example we can count space in RAM, number of instructions, whatever your VM uses every time is executed, save this number and add as fee to next operation the script executes so for example in the next transfer fee will be: fee_of_transfer + consumed_ram_since_last_time So account will need to have enough token in his account for this transfer to succeed. This is of course very very briefly, i didn't checked anything related to it yet in wren or lua and probably will not explore it either now as the solution at least by now will not be pushed to more than just a small proof of concept. It also sounds like will be reinventing the wheel as ethereum VM already handle this in a more natural way from design. |
For the economic question: Every operation that the script triggers must pay the normal BTS fee involved. The creation fee of the script should account for its complexity and duration of being enabled (similar to custom active authority). The script account itself should have BTS balance to pay for the fees of all triggered operations. The script itself could employ a manual check initially if the balance is sufficient, i.e. give the responsibility of atomicity to the script creator. |
Steem is making a feature named "automated actions", I'm not sure what it is: |
@Zapata Thanks a lot for evaluation of our work! |
I am wondering if adoption of Ethereum VM is the right way politically. IMO the EOSIO implementation will receive more adoption due to the massive amount of money floating around. This would allow BitShares to adopt any of the new EOSIO contracts. I have no clue how much more difficult that implemenation is, just wanted to mention it. In general I would prefer to stick to the existing fee schedule, meaning that the script needs to execute any of the existing operations to manipulate chain state, and in that matter also the transaction fee for that. That might be unsuitable for adopting any of the existing virtual machines though. Did you checkout what Steem is doing @OpenLedgerApp ? |
Have the same feelings like @sschiessl-bcp has about Ethereum VM, it is buggy and has big disadvantages also as discussed here https://bitcoin.stackexchange.com/questions/40389/what-advantages-and-disadvantages-does-ethereum-have-over-bitcoin/40735 |
@sschiessl-bcp, @cogutvalera
|
Hm, I see. I guess I don't have a strict personal preference for the used VM, I leave that discussion to others. Is an implementation possible that does not require a separate GAS, but uses existing fee schedule? Furthermore, have you already tried porting existing scripts? What are the normal use cases of such scripts? Are typically all consensus changing operations reflected with one of the BitShares operations? |
|
@OpenLedgerApp : We should not introduce a new currency, because BTS fees have only 5 digits. |
THIS IS FOR VERY PRELIMINARY DISCUSSION ONLY
As a
user
I wantthe ability to install scripts on-chain (let's call them "dumb" contracts)
so thatI can automate sequences of operations on-chain
.Additional Context (optional)
Ideally we would have the ability to write small scripts (turing-completeness not necessary) that leverage conditionals, loops (bounded) the ability to store (a limited amount of) data on-chain and call BitShares platform ops.
Scripts would be installed as sub-accounts under the account creating them and accessed with a sort of name-spacing mechanism for user-friendliness (e.g. account name: clockwork::dividend-script)
Scripts can have different entry points depending on the operation that impacts them to allow for complete functionalities and better performance while we also allow owner accounts to explicitly call methods on them via a new op i.e. call_script_method(script_account,method_name,[params]).
e.g. in pseudocode
The main issue I see here is that the block-producing nodes don't have access to the full database which severely limits functionality. E.g. I can't query the list of holders of a certain asset inside a script for example.
Still there are use cases I can see this kind of functionality being helpful.
CORE TEAM TASK LIST
The text was updated successfully, but these errors were encountered: