- B2B: business to business. Both sides of the transaction are companies (businesses). The most typical case is Alibaba.
- C2C: person to person, for example: Taobao and Renrenche.
- B2C: business to person, for example: Vipshop and Jumei.
- C2B: person to business. Consumers first put forward demand, and then businesses organize production according to that demand, for example: Shangpin Home Collection.
- O2O: online to offline. It combines offline business opportunities with the internet, so the internet becomes a platform for offline transactions, for example: Meituan Waimai and Ele.me.
- B2B2C: business to business to person, for example: Tmall and JD.
- User side
- Home page: product categories, banner carousel, scrolling news, waterfall loading, recommendations, discounts, hot-selling products, and so on.
- User: login (third-party login), registration, logout, self-service functions such as personal information, browsing history, shipping address, and so on.
- Product: category, list, details, search, hot search, search history, add to cart, favorite, follow, comment, and so on.
- Shopping cart: view, edit (change quantity, delete goods, clear all).
- Order: submit order (pay), historical orders, order details, order comments, and so on.
- Admin side
- CRUD of core business entities.
- Scheduled tasks, both periodic and non-periodic, such as processing unpaid orders and raising alarms for abnormal events found during data collection.
- Report functions, such as importing and exporting Excel and PDF files and showing ECharts statistics on the frontend.
- Permission control, such as RBAC, white list, black list, and so on.
- Business flow, such as starting a refund process. Common workflow engines include Activity, Airflow, and Spiff.
- Third-party services, such as maps, SMS, logistics, payment, real-name verification, weather, monitoring, and cloud storage.
First, we need to understand two concepts: SPU (Standard Product Unit) and SKU (Stock Keeping Unit).
- SPU:
iPhone 6s - SKU:
iPhone 6s 64G Gold
Third-party login means using an account from a third-party website, usually a well-known social website, to do login verification. The main purpose is to get related user information from that third-party website. Examples in China are QQ and Weibo, and examples outside China are Google and Facebook. Most third-party login systems use the OAuth protocol. It is an open standard about authorization. The owner of the data tells the system that they agree to let a third-party application enter the system and get the data. The system then creates a short-term access token to replace the password for the third-party application. It is widely used, and version 2.0 is usually used now.
About the difference between a token and a password, we can simply summarize three differences:
- A token is short-term. It will expire automatically, and the user cannot change it by themself. A password is usually valid for a long time. If the user does not change it, it will not change.
- A token can be revoked by the owner of the data, and it becomes invalid immediately. A password usually cannot be revoked by other people.
- A token has a permission scope. For network services, a read-only token is safer than a read-write token. A password usually has full permission.
So, with a token, a third-party application can get permission, but that permission is still controllable at any time and will not endanger system security. This is the advantage of the OAuth protocol.
- After the user opens the client, the client asks the user, who is the resource owner, to give authorization.
- The user agrees to give authorization to the client.
- The client uses the authorization from the previous step to apply for an access token from the authentication server.
- After authenticating the client, the authentication server issues the access token.
- The client uses the access token to apply to the resource server for resources.
- The resource server confirms that the access token is correct and agrees to open the resource to the client.
If Weibo login is used, the specific steps can be found in the "Weibo Login Access" document on the Weibo open platform. If QQ login is used, you first need to register as a QQ Connect developer and pass the review. The specific steps can be found in the QQ Connect access guide and website development process documents.
Tip: On GitBook there is a book called Django Blog Introduction that uses GitHub as an example to introduce third-party account login. You can read it if you are interested.
Usually, when an e-commerce website uses third-party login, it will ask the user to bind the third-party account to the website account, or it will automatically finish account binding according to the third-party account information it gets, such as a mobile phone number.
Cache warm-up means loading data into the cache in advance when the server starts. For this, we can write a subclass of AppConfig in the Django application's apps.py module and override the ready() method, as shown below.
import pymysql
from django.apps import AppConfig
from django.core.cache import cache
SELECT_PROVINCE_SQL = 'select distid, name from tb_district where pid is null'
class CommonConfig(AppConfig):
name = 'common'
def ready(self):
conn = pymysql.connect(host='1.2.3.4', port=3306,
user='root', password='pass',
database='db', charset='utf8',
cursorclass=pymysql.cursors.DictCursor)
try:
with conn.cursor() as cursor:
cursor.execute(SELECT_PROVINCE_SQL)
provinces = cursor.fetchall()
cache.set('provinces', provinces)
finally:
conn.close()Next, you also need to write the following code in the application's __init__.py.
default_app_config = 'common.apps.CommonConfig'Or register the application in the project's settings.py.
INSTALLED_APPS = [
...
'common.apps.CommonConfig',
...
]Use a custom decorator to cache query results.
from pickle import dumps, loads
from django.core.cache import caches
MODEL_CACHE_KEY = 'project:modelcache:%s'
def my_model_cache(key, section='default', timeout=None):
"""Decorator that implements model caching."""
def wrapper1(func):
def wrapper2(*args, **kwargs):
real_key = '%s:%s' % (MODEL_CACHE_KEY % key, ':'.join(map(str, args)))
serialized_data = caches[section].get(real_key)
if serialized_data:
data = loads(serialized_data)
else:
data = func(*args, **kwargs)
cache.set(real_key, dumps(data), timeout=timeout)
return data
return wrapper2
return wrapper1@my_model_cache(key='provinces')
def get_all_provinces():
return list(Province.objects.all())Question 1: Where should the shopping cart of a logged-in user be stored? Where should the shopping cart of a user who is not logged in be stored?
class CartItem(object):
"""An item in the shopping cart."""
def __init__(self, sku, amount=1, selected=False):
self.sku = sku
self.amount = amount
self.selected = selected
@property
def total(self):
return self.sku.price * self.amount
class ShoppingCart(object):
"""Shopping cart."""
def __init__(self):
self.items = {}
self.index = 0
def add_item(self, item):
if item.sku.id in self.items:
self.items[item.sku.id].amount += item.amount
else:
self.items[item.sku.id] = item
def remove_item(self, sku_id):
if sku_id in self.items:
self.items.remove(sku_id)
def clear_all_items(self):
self.items.clear()
@property
def cart_items(self):
return self.items.values()
@property
def cart_total(self):
total = 0
for item in self.items.values():
total += item.total
return totalThe shopping cart of a logged-in user can be stored in the database, and it can first be cached in Redis. The shopping cart of a user who is not logged in can be stored in Cookie, localStorage, or sessionStorage, to reduce the memory cost on the server side.
{
"1001": {"sku": "...", "amount": 1, "selected": true},
"1002": {"sku": "...", "amount": 2, "selected": false},
"1003": {"sku": "...", "amount": 3, "selected": true}
}request.get_signed_cookie('cart')
cart_base64 = base64.base64encode(pickle.dumps(cart))
response.set_signed_cookie('cart', cart_base64)Question 2: After the user logs in, how should the shopping carts be merged? At present, shopping carts in e-commerce applications are almost all persisted, mainly to make it easy to share data across multiple terminals.
Question 1: How should payment information be persisted? Every transaction must be recorded.
Question 2: How should Alipay be integrated? Integrating other platforms is almost similar.
- Ant Financial Open Platform
- Join the platform
- Developer center
- Document center
- SDK integration - PyPI link
- API list
Configuration file:
ALIPAY_APPID = '......'
ALIPAY_URL = 'https://openapi.alipaydev.com/gateway.do'
ALIPAY_DEBUG = FalseGet the payment link and start payment:
# Create the object used to call Alipay
alipay = AliPay(
# The ID assigned when the application is created online
appid=settings.ALIPAY_APPID,
app_notify_url=None,
# Private key of your own application
app_private_key_path=os.path.join(
os.path.dirname(os.path.abspath(__file__)),
'keys/app_private_key.pem'),
# Public key of Alipay
alipay_public_key_path=os.path.join(
os.path.dirname(os.path.abspath(__file__)),
'keys/alipay_public_key.pem'),
sign_type='RSA2',
debug=settings.ALIPAY_DEBUG
)
# Call the operation that gets the payment page
order_info = alipay.api_alipay_trade_page_pay(
out_trade_no='...',
total_amount='...',
subject='...',
return_url='http://...'
)
# Generate the full payment page URL
alipay_url = settings.ALIPAY_URL + '?' + order_info
return JsonResponse({'alipay_url': alipay_url})Through the link returned above, the user can enter the payment page. After payment is finished, the browser will automatically jump back to the project page set in the code above. On that page, the order number (out_trade_no), payment serial number (trade_no), transaction amount (total_amount), and corresponding signature (sign) can be obtained, and then the backend can verify and save the transaction result, as shown below.
# Create the object used to call Alipay
alipay = AliPay(
# The ID assigned when the application is created online
appid=settings.ALIPAY_APPID,
app_notify_url=None,
# Private key of your own application
app_private_key_path=os.path.join(
os.path.dirname(os.path.abspath(__file__)),
'keys/app_private_key.pem'),
# Public key of Alipay
alipay_public_key_path=os.path.join(
os.path.dirname(os.path.abspath(__file__)),
'keys/alipay_public_key.pem'),
sign_type='RSA2',
debug=settings.ALIPAY_DEBUG
)
# Request parameters, assuming this is a POST request,
# include order number, payment serial number, transaction amount, and signature
params = request.POST.dict()
# Call the verification operation
if alipay.verify(params, params.pop('sign')):
# Persist the transactionThe Alipay payment API also provides a series of interfaces such as transaction query, transaction settlement, refund, and refund query. These can be called according to business needs, and they will not be described in detail here.
- Flash sale: a flash sale usually means that very high concurrency must be handled in a very short time. The system needs to bear traffic that may be more than one hundred times the usual traffic in a short time. So flash-sale architecture is a relatively complex problem. Its core ideas are traffic control and performance optimization, and it needs cooperation from every part, from the frontend to the backend. On the frontend, JavaScript can be used to make a countdown, avoid duplicate submission, and limit frequent refresh. Traffic control mainly means allowing only a small part of the traffic to enter the backend service, because only a small number of users can finally succeed in the flash sale. In the physical architecture, cache and message queues can be used to optimize the system. Cache is used because there are many reads and few writes. Inventory can be put in Redis and the
DECRprimitive can be used to reduce stock. Redis can also be used for rate limiting. The reason is similar to limiting how often phone verification codes can be sent. The most important role of the message queue is smoothing traffic peaks and decoupling upstream and downstream nodes. In addition, stateless service design should be used, so that horizontal scaling is easier. - Overselling: suppose the stock of a product is 1. At the same time, user 1 and user 2 buy this product concurrently. After user 1 submits the order, the stock is changed to 0. But user 2, without knowing that, also submits the order, and the stock is changed to -1. This is the overselling problem. There are three common ideas for solving it:
- Pessimistic locking: when querying the quantity of the product, use
select ... for updateto lock the data. In this way, when user 1 checks the inventory, user 2 is blocked and cannot read the inventory quantity. User 2 can continue only after user 1 commits or rolls back the inventory update. This solves the overselling problem. But for products with very high concurrent access, this method performs too badly. In actual development, locking can be considered only when the inventory is smaller than a certain value, but in general this method is not a very good choice. - Optimistic locking: do not lock when querying the quantity. When updating the inventory, require that the quantity of the product must still be the same as the quantity read before. Otherwise, it means another transaction has already updated the inventory, and the request must be sent again.
- Try to reduce inventory directly: combine the query (
select) and update (update) into one SQL operation. When updating inventory, add a condition such asinventory >= purchase_quantityorinventory - purchase_quantity >= 0to thewherecondition. This method requires the transaction isolation level to be read committed.
- Pessimistic locking: when querying the quantity of the product, use
For static resources, you can build your own file server or distributed file server such as FastDFS. But for a normal project, there is usually no need to do this, and the effect may not be the best. We recommend using cloud storage services to manage the static resources of the website. Cloud service providers in China and other countries, such as Amazon, Aliyun, Qiniu, LeanCloud, and Bmob, all provide very good cloud storage services, and the prices are acceptable for ordinary companies. For detailed operations, you can read the official documents, for example the Aliyun OSS developer guide.
- Use the fuzzy query function of the database: the efficiency is low, a full table scan is needed every time, and word segmentation is not supported.
- Use the full-text search function of the database: before MySQL 5.6 it was only suitable for the MyISAM engine. The search operation and other DML operations are coupled inside the database, which may cause the search operation to be very slow. When the amount of data reaches the million level, the performance drops greatly and the query time becomes long.
- Use an open-source search engine: index data and original data are separated. ElasticSearch or Solr can be used to provide an external indexing service. If high-concurrency full-text search is not required, the pure Python Whoosh can also be considered.
ElasticSearch is both a distributed document database and a highly scalable open-source full-text search and analysis engine. It allows the storage, search, and analysis of large amounts of data, and this process is near real time. It is usually used as the underlying engine and technology that powers complex search functions and requirements. Well-known sites such as Wikipedia, Stack Overflow, and GitHub all use ElasticSearch.
The underlying engine of ElasticSearch is the open-source search engine Lucene. But using Lucene directly is very troublesome. You must write code yourself to call its interfaces, and it only supports Java. ElasticSearch is like a complete package around Lucene. It provides REST-style API interfaces and hides programming-language differences through HTTP access. ElasticSearch builds an inverted index. But its built-in tokenizer gives almost no support for Chinese word segmentation, so the elasticsearch-analysis-ik plugin needs to be installed to provide Chinese tokenization.
Besides ElasticSearch, Solr and Whoosh can also be used to provide search engine service. In a Django project, the following solutions can be considered:
haystack(django-haystack/drf-haystack) +whoosh+Jiebahaystack(django-haystack/drf-haystack) +elasticsearchrequests+elasticsearchdjango-elasticsearch-dsl
- Use Docker to install ElasticSearch.
docker pull elasticsearch:7.6.0
docker run -d -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" -e ES_JAVA_OPTS="-Xms512m -Xmx512m" --name es elasticsearch:7.6.0- Enter the
pluginsdirectory in the Docker container.
docker exec -it es /bin/bashyum install -y wget
cd plugins/
mkdir ik
cd ik
wget https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.6.0/elasticsearch-analysis-ik-7.6.0.zip
unzip elasticsearch-analysis-ik-7.6.0.zip
rm -f elasticsearch-analysis-ik-7.6.0.zip
cd ..
mkdir pinyin
cd pinyin
wget https://github.com/medcl/elasticsearch-analysis-pinyin/releases/download/v7.6.0/elasticsearch-analysis-pinyin-7.6.0.zip
unzip elasticsearch-analysis-pinyin-7.6.0.zip
rm -f elasticsearch-analysis-pinyin-7.6.0.zip- Exit the container and restart ElasticSearch.
docker restart es- Test Chinese tokenization.
Request: POST http://1.2.3.4:9200/_analyze
{
"analyzer": "ik_smart",
"text": "中国男足在2022年卡塔尔世界杯预选赛中勇夺小组最后一名"
}- Test pinyin tokenization.
Request: POST http://1.2.3.4:9200/_analyze
{
"analyzer": "pinyin",
"text": "张学友"
}You can search through GET or POST requests. The following example shows searching for goods with the keyword 未来.
-
GET http://120.77.222.217:9200/demo/goods/_search?q=未来Note: Chinese in the URL should be processed as percent-encoding.
Available search parameters in the URL are shown below:
| Parameter | Meaning |
|---|---|
q |
Query string |
analyzer |
Tokenizer used to analyze the query string |
analyze_wildcard |
Whether wildcard or prefix queries are analyzed, default is false |
default_operator |
The relationship between multiple conditions, default is OR, can be changed to AND |
explain |
Include the explanation of the scoring mechanism in the returned result |
fields |
Only return specified columns in the index, separated by commas |
sort |
Field used for sorting, :asc and :desc can be used for ascending and descending order |
timeout |
Timeout length |
from |
Start value of matching results, default is 0 |
size |
Number of matching results, default is 10 |
POST http://120.77.222.217:9200/demo/goods/_search
{
"query": {
"term": {
"type": ""
}
}
}POST search is based on DSL.
The third-party Python library used to connect to ElasticSearch is HayStack. In a Django project, you can use django-haystack. Through HayStack, many search engine services can be connected without changing the code.
pip install django-haystack elasticsearchConfiguration:
INSTALLED_APPS = [
...
'haystack',
...
]
HAYSTACK_CONNECTIONS = {
'default': {
# Engine configuration
'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
# URL of the search engine service
'URL': 'http://1.2.3.4:9200',
# Name of the index library
'INDEX_NAME': 'goods',
},
}
# Automatically generate indexes when data is added, deleted, or updated
HAYSTACK_SIGNAL_PROCESSOR = 'haystack.signals.RealtimeSignalProcessor'Index class:
from haystack import indexes
class GoodsIndex(indexes.SearchIndex, indexes.Indexable):
text = indexes.CharField(document=True, use_template=True)
def get_model(self):
return Goods
def index_queryset(self, using=None):
return self.get_model().objects.all()Edit the template of the text field. It should be placed in templates/search/indexes/demo/goods_text.txt.
{{object.title}}
{{object.intro}}
Configure URL:
urlpatterns = [
# ...
url('search/', include('haystack.urls')),
]Generate the initial index:
python manage.py rebuild_index