Base models¶
Base models describe how cryptoassets.core handles any cryptocurrency on the database level. SQLAlchemy library is used for modeling.
Models are abstract and when you instiate a new cryptocurrency, you inherit from the base classes and set the cryptocurrency specific properties.
Models also specify the core API how to interact with cryptoassets.core
See how to get started interacting with models.
For more information, see coin documentation and how to extend the framework with your own altcoins.
Cryptoasset registry¶
All running cryptoassets are maintained in a coin registry.
Each cryptoasset provides its own Wallet SQLAlchemy model and backend instance which is used to communicate with the network of the cryptoasset.
-
class
cryptoassets.core.coin.registry.
CoinModelDescription
(coin_name, wallet_model_name, address_model_name, account_model_name, transaction_model_name, network_transaction_model_name, address_validator)[source]¶ Describe one cryptocurrency data structures: what SQLAlchemy models and database tables it uses.
The instance of this class is used by
cryptoassets.core.models.CoinDescriptionModel
to build the model relatinoships and foreign keys between the tables of one cryptoasset.Create the description with fully dotted paths to Python classes.
Parameters: coin_name – Name of this coin, lowercase acronym -
Wallet
¶ Get wallet model class.
-
Address
¶ Get address model class.
-
Account
¶ Get account model class.
-
NetworkTransaction
¶ Get network transaction model class.
-
Transaction
¶ Get transaction model class.
-
-
class
cryptoassets.core.coin.registry.
Coin
(coin_description, backend=None, max_confirmation_count=15, testnet=False)[source]¶ Describe one cryptocurrency setup.
Binds cryptocurrency to its backend and database models.
We also carry a flag if we are running in testnet or not. This affects address validation.
Create a binding between asset models and backend.
Parameters: - coin_description –
cryptoassets.core.coin.registry.CoinModelDescription
- testnet – Are we running a testnet node or real node.
- backend –
cryptoassets.core.backend.base.CoinBackend
-
backend
= None¶ Subclass of
cryptoassets.core.backend.base.CoinBackend
.
-
name
= None¶ Lowercase acronym name of this asset
-
max_confirmation_count
= None¶ This is how many confirmations
tools.confirmationupdate
tracks for each network transactions, both incoming and outgoing, until we consider it “closed” and stop polling backend for updates.
-
address_model
¶ Property to get SQLAlchemy model for address of this cryptoasset.
Subclass of
cryptoassets.core.models.GenericAddress
.
-
transaction_model
¶ Property to get SQLAlchemy model for transaction of this cryptoasset.
Subclass of
cryptoassets.core.models.GenericTransaction
.
-
account_model
¶ Property to get SQLAlchemy model for account of this cryptoasset.
Subclass of
cryptoassets.core.models.GenericAccount
.
-
wallet_model
¶ Property to get SQLAlchemy model for account of this cryptoasset.
Subclass of
cryptoassets.core.models.GenericWallet
.
-
network_transaction_model
¶ Property to get SQLAlchemy model for account of this cryptoasset.
Subclass of
cryptoassets.core.models.GenericWallet
.
- coin_description –
-
class
cryptoassets.core.coin.registry.
CoinRegistry
[source]¶ Holds data of set up cryptocurrencies.
Usually you access this through
cryptoasssets.core.app.CryptoassetsApp.coins
instance.Example:
cryptoassets_app = CryptoassetsApp() # ... setup ... bitcoin = cryptoassets_app.coins.get("btc) print("We are running bitcoin with backend {}".format(bitcoin.backend))
Default models¶
Default cryptocurrency names and their models.
-
cryptoassets.core.coin.defaults.
COIN_MODEL_DEFAULTS
= {'ltc': 'cryptoassets.core.coin.litecoin.models', 'aby': 'cryptoassets.core.coin.applebyte.models', 'btc': 'cryptoassets.core.coin.bitcoin.models', 'doge': 'cryptoassets.core.coin.dogecoin.models'}¶ This is the default mapping between the three-letter coin acronyms and their SQLAlchemy model presentations. If you want to use your own database models you can override any of these in your configuration.
Model API conventions¶
The following conventions are followed in the model API
Model discovery¶
- Abstract base classes are called GenericXxx like GenericWallet.
- Actual class implementation is in
coin
module, e.g.cryptoassets.core.coin.bitcoin.models.BitcoinWallet
. - You do not access the model classes directly, but through configured assets registry. E.g. to get a hold of
BitcoinWallet
class you doWallet = cryptoassets_app.coins.get("btc").coin_model
. - The usual starting point for the calls is to get or create
cryptoassets.core.models.GenericWallet
instance. Check outcryptoassets.core.models.GenericWallet.get_or_create_by_name()
.
Session lifecycle¶
- API tries to use the SQLAlchemy database session of the object if possible:
Session.object_session(self)
. If not, session must be explicitly given and you get your session inside a helper closure function decorated bycryptoassets.core.utils.conflictresolver.ConflictResolver.managed_transaction()
. This way we guarantee graceful handling of transaction conflicts. - API never does
session.flush()
orsession.commit()
- API will do
session.add()
for newly created objects
Model classes¶
Below are the base classes for models. All cryptoassets have the same API as described these models.
Account¶
-
class
cryptoassets.core.models.
GenericAccount
[source]¶ An account within the wallet.
We associate addresses and transactions to one account.
The accountn can be owned by some user (user’s wallet), or it can be escrow account or some other form of automatic transaction account.
The transaction between the accounts of the same wallet are internal and happen off-blockhain.
A special account is reserved for network fees caused by outgoing transactions.
-
NETWORK_FEE_ACCOUNT
= 'Network fees'¶ Special label for an account where wallet will put all network fees charged by the backend
-
id
= Column(None, Integer(), table=None, primary_key=True, nullable=False)¶ Running counter used in foreign key references
-
name
= Column(None, String(length=255), table=None)¶ Human-readable name for this account
-
created_at
= Column(None, DateTime(), table=None, default=ColumnDefault(<function ColumnDefault._maybe_wrap_callable.<locals>.<lambda> at 0x7f65967e5840>))¶ When this account was created
-
updated_at
= Column(None, DateTime(), table=None, onupdate=ColumnDefault(<function ColumnDefault._maybe_wrap_callable.<locals>.<lambda> at 0x7f65967e58c8>))¶ Then the balance was updated, or new address generated
-
balance
= Column(None, Numeric(precision=21, scale=8), table=None, nullable=False, default=ColumnDefault(0))¶ Available internal balance on this account NOTE: Accuracy checked for bitcoin only
-
pick_next_receiving_address_label
()[source]¶ Generates a new receiving address label which is not taken yet.
Some services, like block.io, requires all receiving addresses to have an unique label. We use this helper function in the situations where it is not meaningful to hand-generate labels every time.
Generated labels are not user-readable, they are only useful for admin and accounting purposes.
-
Address¶
-
class
cryptoassets.core.models.
GenericAddress
(**kwargs)[source]¶ The base class for cryptocurrency addresses.
The address can represent a
- Receiving address in our system. In this case we have account set to non-NULL.
- External address outside our system. In this account is set to NULL. This address has been referred in outgoing broadcast (XXX: subject to change)
We can know about receiving addresses which are addresses without our system where somebody can deposit cryptocurrency. We also know about outgoing addresses where somebody has sent cryptocurrency from our system. For outgoing addresses
wallet
reference is null.Warning
Some backends (block.io) enforce that receiving address labels must be unique across the system. Other’s don’t. Just bear this in mind when creating address labels. E.g. suffix them with a timetamp to make them more unique.
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs
.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
-
id
= Column(None, Integer(), table=None, primary_key=True, nullable=False)¶ Running counter used in foreign key references
-
address
= Column(None, String(length=127), table=None, nullable=False)¶ The string presenting the address label in the network
-
label
= Column(None, String(length=255), table=None)¶ Human-readable label for this address. User for the transaction history listing of the user.
-
balance
= Column(None, Numeric(precision=21, scale=8), table=None, nullable=False, default=ColumnDefault(0))¶ Received balance of this address. Only confirmed deposits count, filtered by GenericConfirmationTransaction.confirmations. For getting other balances, check
get_balance_by_confirmations()
. NOTE: Numeric Accuracy checked for Bitcoin only ATM
-
archived_at
= Column(None, DateTime(), table=None)¶ Archived addresses are no longer in active incoming transaction polling and may not appear in the user wallet list
-
get_received_transactions
(external=True, internal=True)[source]¶ Get all transactions this address have received, both internal and external deposits.
-
get_balance_by_confirmations
(confirmations=0, include_internal=True)[source]¶ Calculates address’s received balance of all arrived incoming transactions where confirmation count threshold is met.
By default confirmations is zero, so we get unconfirmed balance.
Note
This is all time received balance, not balance left after spending.
TODO: Move to its own subclass
Parameters: confirmations – Confirmation count as threshold
Transaction¶
-
class
cryptoassets.core.models.
GenericTransaction
(**kwargs)[source]¶ A transaction between accounts, incoming transaction or outgoing transaction.
Transactions can be classified as following:
Deposit: Incoming, external, transaction from cryptocurrency network.
- Has
network_transaction
set. - Has
receiving_account
set. - No
sending_account
- Has
Broadcast: Outgoign, external, transaction to cryptocurrency network.
- Has
network_transaction
set. - Has
receiving_account
set. - No
receiving_account
- Has
Internal transactions
- Which are not visible outside our system.
- have both
sending_account
andreceiving_account
set. network_transaction
is null- Internal transactions can be further classified as:
ìnternal
(normal between accounts),balance_import
(initial wallet import to system) andnetwork_fee
(fees accounted to the network fee account when transaction was broadcasted)
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs
.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
-
id
= Column(None, Integer(), table=None, primary_key=True, nullable=False)¶ Running counter used in foreign key references
-
created_at
= Column(None, DateTime(), table=None, default=ColumnDefault(<function ColumnDefault._maybe_wrap_callable.<locals>.<lambda> at 0x7f65967f42f0>))¶ When this transaction become visible in our database
-
credited_at
= Column(None, DateTime(), table=None)¶ When the incoming transaction was credited on the account. For internal transactions it is instantly. For external transactions this is when the confirmation threshold is exceeded.
-
processed_at
= Column(None, DateTime(), table=None)¶ When this transaction was processed by the application. For outgoing transactions this is the broadcasting time. For incoming transactions, your application may call
mark_as_processed
to mark it has handled the transaction.
-
amount
= Column(None, Numeric(precision=21, scale=8), table=None)¶ Amount in the cryptocurrency minimum unit Note: Accuracy checked for Bitcoin only
-
state
= Column(None, Enum('pending', 'broadcasted', 'incoming', 'processed', 'internal', 'network_fee', 'balance_import', name='transaction_state'), table=None, nullable=False)¶ Different states this transaction can be
pending: outgoing transaction waiting for the broadcast
broadcasted: outgoing transaction has been sent to the network
incoming: we see the transaction incoming to our system, but the confirmation threshold is not exceeded yet processed: the application marked this transaction as handled and cryptoassets.core stops trying to notify your application about the transaction
internal: This transaction was between the accounts within one of our wallets
network_fee: When the transaction has been broadcasted, we create an internal transaction to account the occured network fees
-
label
= Column(None, String(length=255), table=None)¶ Human readable label what this transaction is all about. Must be unique for each account
-
txid
¶ Return txid of associated network transaction (if any).
Shortcut for
self.network_transaction.txid
.
NetworkTransaction¶
-
class
cryptoassets.core.models.
GenericNetworkTransaction
(**kwargs)[source]¶ A transaction in cryptocurrencty networkwhich is concern of our system.
External transactions can be classified as
- Deposits: incoming transactions to our receiving addresses
- Broadcasts: we are sending out currency to the network
If our intenal transaction (
cryptoassets.core.models.Transaction
) has associated network transaction, it’stransaction.network_transaction
reference is set. Otherwise transactions are internal transactions and not visible in blockchain.Note
NetworkTransaction does not have reference to wallet. One network transaction may contain transfers to many wallets.
Handling incoming deposit transactions
For more information see
cryptoassets.core.backend.transactionupdater
andcryptoassets.core.tools.confirmationupdate
.Broadcasting outgoing transactions
Broadcast constructs an network transaction and bundles any number of outgoing pending transactions to it. During the broadcast, one can freely bundle transactions together to lower the network fees, or mix transactions for additional privacy.
Broadcasts are constructed by Cryptoassets helper service which will periodically scan for outgoing transactions and construct broadcasts of them. After constructing, broadcasting is attempted. If the backend, for a reason or another, fails to make a broadcast then this broadcast is marked as open and must be manually vetted to succeeded or failed.
For more information see
cryptoassets.core.tools.broadcast
.A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs
.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
-
id
= Column(None, Integer(), table=None, primary_key=True, nullable=False)¶ Running counter used in foreign key references
-
created_at
= Column(None, DateTime(), table=None, default=ColumnDefault(<function ColumnDefault._maybe_wrap_callable.<locals>.<lambda> at 0x7f65967fbae8>))¶ When this transaction become visible in our database
-
txid
= Column(None, String(length=255), table=None)¶ Network transaction has associated with this transaction. E.g. Bitcoin transaction hash.
-
transaction_type
= Column(None, Enum('deposit', 'broadcast', name='network_transaction_type'), table=None, nullable=False)¶ Is this transaction incoming or outgoing from our system
-
opened_at
= Column(None, DateTime(), table=None)¶ When broadcast was marked as outgoing
-
closed_at
= Column(None, DateTime(), table=None)¶ When broadcast was marked as sent
-
classmethod
get_or_create_deposit
(session, txid)[source]¶ Get a hold of incoming transaction.
Returns: tuple(Instance of cryptoassets.core.models.GenericNetworkTransaction
., bool created)
-
class
cryptoassets.core.models.
GenericConfirmationNetworkTransaction
(**kwargs)[source]¶ Mined transaction which receives “confirmations” from miners in blockchain.
This is a subtype of
GenericNetworkTransaction
with confirmation counting abilities.A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs
.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
-
confirmations
= Column(None, Integer(), table=None, nullable=False, default=ColumnDefault(-1))¶ How many miner confirmations this tx has received. The value is
-1
until the transaction is succesfully broadcasted, after which is it0
-
confirmation_count
= 3¶ How many confirmations to wait until the transaction is set as confirmed. TODO: Make this configurable.
-
Wallet¶
-
class
cryptoassets.core.models.
GenericWallet
[source]¶ A generic wallet implemetation.
Inside the wallet there is a number of accounts.
We support internal transaction between the accounts of the same wallet as off-chain transactions. If you call
send()``for the address which is managed by the same wallet, an internal transaction is created by ``send_internal()
.-
id
= Column(None, Integer(), table=None, primary_key=True, nullable=False)¶ Running counter used in foreign key references
-
name
= Column(None, String(length=255), table=None)¶ The human-readable name for this wallet. Only used for debugging purposes.
-
created_at
= Column(None, Date(), table=None, default=ColumnDefault(<function ColumnDefault._maybe_wrap_callable.<locals>.<lambda> at 0x7f65967f4bf8>))¶ When this wallet was created
-
updated_at
= Column(None, Date(), table=None, onupdate=ColumnDefault(<function ColumnDefault._maybe_wrap_callable.<locals>.<lambda> at 0x7f65967f4c80>))¶ Last time when the balance was updated or new receiving address created.
-
balance
= Column(None, Numeric(precision=21, scale=8), table=None)¶ The total balance of this wallet in the minimum unit of cryptocurrency NOTE: accuracy checked for Bitcoin only
-
classmethod
get_by_id
(session, wallet_id)[source]¶ Returns an existing wallet instance by its id.
Returns: Wallet instance
-
classmethod
get_or_create_by_name
(name, session)[source]¶ Returns a new or existing instance of a named wallet.
Returns: Wallet instance
-
get_or_create_network_fee_account
()[source]¶ Lazily create the special account where we account all network fees.
This is for internal bookkeeping only. These fees MAY be charged from the users doing the actual transaction, but it must be solved on the application level.
-
create_receiving_address
(account, label=None, automatic_label=False)[source]¶ Creates a new receiving address.
All incoming transactions on this address are put on the given account.
The notifications for transctions to the address might not be immediately available after the address creation depending on the backend. For example, with block.io you need to wait some seconds before it is safe to send anything to the address if you wish to receive the wallet notification.
Parameters: - account – GenericAccount object
- label – Label for this address - must be human-readable
Returns: GenericAddress object
-
get_or_create_external_address
(address)[source]¶ Create an accounting entry for an address which is outside our system.
When we send out external transactions, they go to these address entries. These addresses do not have wallet or account connected to our system.
Parameters: address – Address as a string
-
send
(from_account, receiving_address, amount, label, force_external=False, testnet=False)[source]¶ Send the amount of cryptocurrency to the target address.
If the address is hosted in the same wallet do the internal send with
cryptoassets.core.models.GenericWallet.send_internal()
, otherwise go through the public blockchain withcryptoassets.core.models.GenericWallet.send_external()
.Parameters: - from_account – The account owner from whose balance we
- receiving_address – Receiving address as a string
- amount – Instance of Decimal
- label – Recorded text to the sending wallet
- testnet – Assume the address is testnet address. Currently not used, but might affect address validation in the future.
- force_external – Set to true to force the transaction go through the network even if the target address is in our system.
Returns: Transaction object
-
add_address
(account, label, address)[source]¶ Adds an external address under this wallet, under this account.
There shouldn’t be reason to call this directly, unless it is for testing purposes.
Parameters: - account – Account instance
- address – Address instance
-
get_account_by_address
(address)[source]¶ Check if a particular address belongs to receiving address of this wallet and return its account.
This does not consider bitcoin change addresses and such.
Returns: Account instance or None if the wallet doesn’t know about the address
-
get_pending_outgoing_transactions
()[source]¶ Get the list of outgoing transactions which have not been associated with any broadcast yet.
-
get_receiving_addresses
(archived=False)[source]¶ Get all receiving addresses for this wallet.
This is mostly used by the backend to get the list of receiving addresses to monitor for incoming transactions on the startup.
Parameters: expired – Include expired addresses
-
get_deposit_transactions
()[source]¶ Get all deposit transactions to this wallet.
These are external incoming transactions, both unconfirmed and confirmed.
Returns: SQLAlchemy query of Transaction model
-
get_active_external_received_transcations
()[source]¶ Return unconfirmed transactions which are still pending the network confirmations to be credited.
Returns: SQLAlchemy query of Transaction model
-
refresh_account_balance
(account)[source]¶ Refresh the balance for one account.
If you have imported any addresses, this will recalculate balances from the backend.
TODO: This method will be replaced with wallet import.
TODO: This screws ups bookkeeping, so DON’T call this on production. It doesn’t write fixing entries yet.
Parameters: account – GenericAccount instance
-
send_internal
(from_account, to_account, amount, label, allow_negative_balance=False)[source]¶ Tranfer currency internally between the accounts of this wallet.
Parameters: - from_account – GenericAccount
- to_account – GenericAccount
- amount – The amount to transfer in wallet book keeping unit
-
send_external
(from_account, to_address, amount, label, testnet=False)[source]¶ Create a new external transaction and put it to the transaction queue.
When you send cryptocurrency out from the wallet, the transaction is put to the outgoing queue. Only after you broadcast has been performed (
cryptoassets.core.tools.broadcast
) the transaction is send out to the network. This is to guarantee the system responsiveness and fault-tolerance, so that outgoing transactions are created even if we have temporarily lost the connection with the cryptocurrency network. Broadcasting is usually handled by cryptoassets helper service.Parameters: - from_account – Instance of
cryptoassets.core.models.GenericAccount
- to_address – Address as a string
- amount – Instance of Decimal
- label – Recorded to the sending wallet history
- testnet – to_address is a testnet address
Returns: Instance of
cryptoassets.core.models.GenericTransaction
- from_account – Instance of
-
charge_network_fees
(broadcast, fee)[source]¶ Account network fees due to transaction broadcast.
By default this creates a new accounting entry on a special account (GenericAccount.NETWORK_FEE_ACCOUNT) where the network fees are put.
Parameters: - txs – Internal transactions participating in send
- txid – External transaction id
- fee – Fee as the integer
-
refresh_total_balance
()[source]¶ Make the balance to match with the actual backend.
This is only useful for send_external() balance checks. Actual address balances will be out of sync after calling this (if the balance is incorrect).
-
deposit
(ntx, address, amount, extra=None)[source]¶ Informs the wallet updates regarding external incoming transction.
This method should be called by the coin backend only.
Write the transaction to the database. Notify the application of the new transaction status. Wait for the application to mark the transaction as processed.
Note that we may receive the transaction many times with different confirmation counts.
Parameters: - ntx – Associated
cryptoassets.core.models.NetworkTransaction
- address – Address as a string
- amount – Int, as the basic currency unit
- extra – Extra variables to set on the transaction object as a dictionary. (Currently not used)
Returns: tuple (Account instance, new or existing Transaction object, credited boolean)
- ntx – Associated
-
Validation¶
Coin models support pluggable address validators.
We provide some validators just to make sure we don’t write bad outgoing transactions to our database.
-
class
cryptoassets.core.coin.validate.
AddressValidator
[source]¶ Define address validation interface.
You should not call this directly, instead use
cryptoassets.core.coin.registry.Coin.validate_address()
.