Skip to content

Commit

Permalink
Check in TPC-C runner and add BRAD driver (#490)
Browse files Browse the repository at this point in the history
* Check in a fork of the py-tpcc code

* Python 2 to 3 fixes

* Black reformat

* Implement BradDriver with delivery and new_order

* Implement order_status, payment, and stock_level
  • Loading branch information
geoffxy authored Apr 14, 2024
1 parent a7c5988 commit 940749a
Show file tree
Hide file tree
Showing 28 changed files with 6,142 additions and 0 deletions.
32 changes: 32 additions & 0 deletions workloads/chbenchmark/py-tpcc/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Python TPC-C implementation

Source: https://github.com/apavlo/py-tpcc at commit
`db36d72dfbb6bd800d257279be9bbc1a22095ff9`.

See the individual files for the license. The license comment from `tpcc.py` is
shown below.

> -----------------------------------------------------------------------
> Copyright (C) 2011
> Andy Pavlo
> http:##www.cs.brown.edu/~pavlo/
>
> Permission is hereby granted, free of charge, to any person obtaining
> a copy of this software and associated documentation files (the
> "Software"), to deal in the Software without restriction, including
> without limitation the rights to use, copy, modify, merge, publish,
> distribute, sublicense, and/or sell copies of the Software, and to
> permit persons to whom the Software is furnished to do so, subject to
> the following conditions:
>
> The above copyright notice and this permission notice shall be
> included in all copies or substantial portions of the Software.
>
> THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
> IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
> OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
> ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
> OTHER DEALINGS IN THE SOFTWARE.
> -----------------------------------------------------------------------
52 changes: 52 additions & 0 deletions workloads/chbenchmark/py-tpcc/README.original
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
+ ----------------------------------------------- +
+ Python TPC-C +
+ ----------------------------------------------- +


The basic idea is that you will need to create a new driver file that
implements the functions defined in "abstractdriver.py". One function will
load in the tuples into your database for a given table. Then there are five
separate functions that execute the given transaction based on a set of input
parameters. All the work for generating the tuples and the input parameters
for the transactions has been done for you.

Here's what you need to do to get started:

(1) Download the source code from Github:

https://github.com/apavlo/py-tpcc/tree/master/pytpcc

(2) Create a new file in the 'drivers' directory for your system that follows
the proper naming convention. For example, if your system is 'MongoDB', then
your new file will be called 'mongodbdriver.py' and that file will contain a
new class called 'MongodbDriver' (note the capitalization).

(3) Inside your class you will need to implement the required functions of
defined in AbstractDriver. There is documentation on what these need to do
also available on Github:

https://github.com/apavlo/py-tpcc/wiki

(3) Try running your system. I would start by defining the configuration file
that gets returned with by the 'makeDefaultConfig' function in your driver and
then implement the data loading part first, since that will guide how you
actually execute the transactions. Using 'MongoDB' as an example again, you
can print out the driver's configuration dict to a file:

$ python ./tpcc.py --print-config mongodb > mongodb.config

Make any changes you need to 'mongodb.config' (e.g., passwords, hostnames).
Then test the loader:

$ python ./tpcc.py --no-execute --config=mongodb.config mongodb

You can use the CSV driver if you want to see what the data or transaction
input parameters will look like. The following command will dump out just the
input to the driver's functions to files in /tmp/tpcc-*

$ python ./tpcc.py csv

You can also look at my SqliteDriver implementation to get an idea of what
your transaction implementation functions need to do:

https://github.com/apavlo/py-tpcc/blob/master/pytpcc/drivers/sqlitedriver.py
3 changes: 3 additions & 0 deletions workloads/chbenchmark/py-tpcc/pytpcc/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
*.pyc
.#kate-*
*.config
17 changes: 17 additions & 0 deletions workloads/chbenchmark/py-tpcc/pytpcc/CONFIG_EXAMPLE
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# HypertableDriver Configuration File
# Created 2011-05-02 01:43:37.859545
[hypertable]

# hostname
host = localhost

# namespace name
namespace = tpcc

# port
port = 38080

#clientnodes splited by spaces
clients =u1 u2 192.168.3.21
#directories of the code on the client node
path =./code/tpcc/py-tpcc/mtpcc
8 changes: 8 additions & 0 deletions workloads/chbenchmark/py-tpcc/pytpcc/README_v1.1
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
1. 3 newly added files message.py worker.py coordinator.py. Copy them into the old pytpcc directory.
2. Coordinator is the main part. Use it like the old tpcc.py. e.g., python coordinator.py --config hypertable.config --clientprocs 5 hypertable
3. Old argument --clients is replaced with --clientprocs, which specifies how many worker processes you want to run on each client node.
4. All clientnodes(name or ip) must be specified in the configure file.
5. The directory of the code on the client side should be specified in the configure file,too. The default address is user's home address, which is default using ssh.
It should remain the same for each client node, which is not a problem for now.
6. Execnet python module should be installed on each client. Here is how to install it. http://codespeak.net/execnet/install.html

1 change: 1 addition & 0 deletions workloads/chbenchmark/py-tpcc/pytpcc/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
#
168 changes: 168 additions & 0 deletions workloads/chbenchmark/py-tpcc/pytpcc/constants.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
# -*- coding: utf-8 -*-
# -----------------------------------------------------------------------
# Copyright (C) 2011
# Andy Pavlo
# http://www.cs.brown.edu/~pavlo/
#
# Original Java Version:
# Copyright (C) 2008
# Evan Jones
# Massachusetts Institute of Technology
#
# Permission is hereby granted, free of charge, to any person obtaining
# a copy of this software and associated documentation files (the
# "Software"), to deal in the Software without restriction, including
# without limitation the rights to use, copy, modify, merge, publish,
# distribute, sublicense, and/or sell copies of the Software, and to
# permit persons to whom the Software is furnished to do so, subject to
# the following conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
# IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
# OTHER DEALINGS IN THE SOFTWARE.
# -----------------------------------------------------------------------

MONEY_DECIMALS = 2

# Item constants
NUM_ITEMS = 100000
MIN_IM = 1
MAX_IM = 10000
MIN_PRICE = 1.00
MAX_PRICE = 100.00
MIN_I_NAME = 14
MAX_I_NAME = 24
MIN_I_DATA = 26
MAX_I_DATA = 50

# Warehouse constants
MIN_TAX = 0
MAX_TAX = 0.2000
TAX_DECIMALS = 4
INITIAL_W_YTD = 300000.00
MIN_NAME = 6
MAX_NAME = 10
MIN_STREET = 10
MAX_STREET = 20
MIN_CITY = 10
MAX_CITY = 20
STATE = 2
ZIP_LENGTH = 9
ZIP_SUFFIX = "11111"

# Stock constants
MIN_QUANTITY = 10
MAX_QUANTITY = 100
DIST = 24
STOCK_PER_WAREHOUSE = 100000

# District constants
DISTRICTS_PER_WAREHOUSE = 10
INITIAL_D_YTD = 30000.00 # different from Warehouse
INITIAL_NEXT_O_ID = 3001

# Customer constants
CUSTOMERS_PER_DISTRICT = 3000
INITIAL_CREDIT_LIM = 50000.00
MIN_DISCOUNT = 0.0000
MAX_DISCOUNT = 0.5000
DISCOUNT_DECIMALS = 4
INITIAL_BALANCE = -10.00
INITIAL_YTD_PAYMENT = 10.00
INITIAL_PAYMENT_CNT = 1
INITIAL_DELIVERY_CNT = 0
MIN_FIRST = 6
MAX_FIRST = 10
MIDDLE = "OE"
PHONE = 16
MIN_C_DATA = 300
MAX_C_DATA = 500
GOOD_CREDIT = "GC"
BAD_CREDIT = "BC"

# Order constants
MIN_CARRIER_ID = 1
MAX_CARRIER_ID = 10
# HACK: This is not strictly correct, but it works
NULL_CARRIER_ID = 0
# o_id < than this value, carrier != null, >= -> carrier == null
NULL_CARRIER_LOWER_BOUND = 2101
MIN_OL_CNT = 5
MAX_OL_CNT = 15
INITIAL_ALL_LOCAL = 1
INITIAL_ORDERS_PER_DISTRICT = 3000

# Used to generate new order transactions
MAX_OL_QUANTITY = 10

# Order line constants
INITIAL_QUANTITY = 5
MIN_AMOUNT = 0.01

# History constants
MIN_DATA = 12
MAX_DATA = 24
INITIAL_AMOUNT = 10.00

# New order constants
INITIAL_NEW_ORDERS_PER_DISTRICT = 900

# TPC-C 2.4.3.4 (page 31) says this must be displayed when new order rolls back.
INVALID_ITEM_MESSAGE = "Item number is not valid"

# Used to generate stock level transactions
MIN_STOCK_LEVEL_THRESHOLD = 10
MAX_STOCK_LEVEL_THRESHOLD = 20

# Used to generate payment transactions
MIN_PAYMENT = 1.0
MAX_PAYMENT = 5000.0

# Indicates "brand" items and stock in i_data and s_data.
ORIGINAL_STRING = "ORIGINAL"

# Table Names
TABLENAME_ITEM = "ITEM"
TABLENAME_WAREHOUSE = "WAREHOUSE"
TABLENAME_DISTRICT = "DISTRICT"
TABLENAME_CUSTOMER = "CUSTOMER"
TABLENAME_STOCK = "STOCK"
TABLENAME_ORDERS = "ORDERS"
TABLENAME_NEW_ORDER = "NEW_ORDER"
TABLENAME_ORDER_LINE = "ORDER_LINE"
TABLENAME_HISTORY = "HISTORY"

ALL_TABLES = [
TABLENAME_ITEM,
TABLENAME_WAREHOUSE,
TABLENAME_DISTRICT,
TABLENAME_CUSTOMER,
TABLENAME_STOCK,
TABLENAME_ORDERS,
TABLENAME_NEW_ORDER,
TABLENAME_ORDER_LINE,
TABLENAME_HISTORY,
]


# Transaction Types
def enum(*sequential, **named):
enums = dict(map(lambda x: (x, x), sequential))
# dict(zip(sequential, range(len(sequential))), **named)
return type("Enum", (), enums)


TransactionTypes = enum(
"DELIVERY",
"NEW_ORDER",
"ORDER_STATUS",
"PAYMENT",
"STOCK_LEVEL",
)
Loading

0 comments on commit 940749a

Please sign in to comment.