In the first year of the big model, omnipotent Taobao had omnipotent AI.
Mengchen originated from Aofei Temple.Quantum bit | WeChat official account QbitAI
This year’s Double Eleven, in addition to buyers, sellers and platforms, there is also a fourth participant:
AI assistant.
Taobao Tmall platform, merchants have called AI more than 1.5 billion times in the preparation period and sales period of Double Eleven.
In terms of buyers, the cumulative experience of AI assistant Taobao has exceeded 10 million, and the average number of questions asked by highly active people has exceeded 8 times a day.
The most exaggerated data is that"The maximum number of questions asked by a single user exceeds 4,000 times", this is not looking for goods at all, what to buy all listen to AI?
Taobao Tmall’s various AI applications this time, it is a hundred flowers.
If you don’t know what to buy, you can ask AI. If you choose which one, you can ask AI. If you choose how to place an order and which activities are affordable, you can also ask AI.
That’s not all. Sellers can find AI when they open new stores quickly, AI when they put goods on shelves in large quantities, and AI when they advertise.
……
The big model broke out for nearly a year, constantly transforming people’s work and online entertainment.
Infiltrating into e-commerce shopping can be regarded as connecting with offline life such as food, clothing, housing and transportation with a broader space.
AI shopping new experience
AI assistant Taobao asked that it has been opened to all Taobao users.You can directly enter "Taobao Ask" in the search box to get there..
At first glance, similar to other chatbot products, the main body is a dialog box.
In fact, I found that it not only knows the goods, but also knows you.
If there are coupons to make up for, you can ask AI to recommend snacks.
There is no need to add additional requirements when asking questions, and it can also recommend products that meet your preferences from historical orders, shopping carts and other data.
For each recommended product, AI will summarize the characteristics of the product itself, the suitable people or occasions, and the reasons for recommendation.
Next, click on the product card, and you can directly select the specifications and tastes to join the shopping cart.
In addition to asking questions directly, there are some built-in function templates in Taobao Ask.
"Choose goods with me" specializes in the difficulty of choosing, and the advantages and disadvantages of choosing two commodities are clearly compared.
"Wedding planner", "travel planner" and "senior shopping guide" are all specially optimized for the corresponding scenes, and the overall plan+product recommendation of each link are presented in one breath.
"Little experts in life" and "experts in food" can purchase all the necessary materials with one click on the basis of solving daily problems.
Finally, the "soul writer" is for users who like to share shopping experiences.
Here, let’s introduce the game of "choosing goods with me" to compare goods: choose two uncertain goods directly from browsing records or shopping carts.
Even if it is different kinds of laundry detergent and laundry beads, AI can help you analyze the similarities and differences clearly.
Of course, if you want to compare clothes and how to match them well, Taobao can’t help you for the time being.
At this time, please bring out another function."Taobao fitting", or direct search can be found.
Optimistic about which one can be changed with a click, and you can match your own outfit or choose a suit, which is a miracle warmth of the real version.
When it comes to matching hair, face, body and legs, you can also upload your own full-body photos and set your own height and weight instead of AI models, and you can try on tens of millions online without going out and spending money.
You can click the compare button and compare it repeatedly with the state when you take pictures, which is also a very practical gameplay.
In addition to the above introduction, Taobao Tmall has also prepared"My adorable pet"Make a digital avatar for your pet through AIGC technology;"Very homely", upload room photos, online design and decoration programs and more.
They are all searched directly in the search bar, and the space is limited, so it is left to interested readers to experience it themselves.
Merchants also have a special"Intelligent Management Tools for Taobao Merchants"From uploading the first photo of the product, the store was named, the Logo was generated, the attributes of the product were automatically identified by only one photo, and it was put on the shelves and decorated in the store in one go, saving a lot of manual operations.
In addition, there is the function of expanding the size of commodity materials at will, which is suitable for various display occasions, greatly reducing the processing cost of materials and avoiding manual rework.
In the end, judging from the data of the re-opening, this year’s Double Eleven is the session in which new brands, new businesses and small and medium-sized businesses participated the most.
As usual, unfamiliar platform functions and unskilled operation will bring a lot of trouble, but this year, with the addition of new technologies, the threshold for operating stores has been completely lowered.
In less than half a month, Taobao merchants’ intelligent management tool line provided 10w+ AI hair distribution service for women’s clothing merchants, and the hair distribution time was reduced by 25% compared with traditional hair distribution.
Copilot, a centralized e-commerce business developed based on the big language model technology, has provided tens of thousands of businesses with a variety of business assistance capabilities, such as business knowledge question and answer, business tool call, copy generation, etc., and has improved business efficiency for more than 50w times.
The first large-scale application of AI technology in the field of e-commerce
Counting, the release of ChatGPT is close to the first anniversary, and the rise of AIGC Wensheng Map is even earlier.
Every technological breakthrough during this period has set off a wave of starting a business or transforming existing businesses. Until this year’s Double Eleven period, AI finally achieved large-scale application in the field of e-commerce, which is of extraordinary significance:
It is not only used for internal processes of enterprises, but also for end users. Moreover, it is an end-user with completely different characteristics and needs from buyers and sellers.
This involves the integration and innovation of two major technical routes: the big model and the AIGC Wensheng map. It can be observed how AI crosses the gap from technology to application, and it can also bring inspiration to more industries.
Let’s talk about the earlier rise of AIGC Wensheng map. In August, 2022, Stable Diffusion was open source, and the demand for computing power was small, which opened the prelude to the commercialization of AIGC.
But the weakness of Stable Diffusion is also exposed in practice:
First of all, the generated images are uncontrollable, especially in the e-commerce field where Taobao Tmall is located, which requires extremely high accuracy in the presentation of commodity subjects.
Secondly, it is not easy to master the complex and complicated prompt skills, and many parameters that need to be adjusted increase the learning cost.
The turnaround will soon appear. In January, ControlNet was born, and it was skillfully generated by adding auxiliary conditions, which solved the controllability problem to some extent and was known as "changing the rules of the game".
At this time, the master can already use the combination of Stable Diffusion+ControlNet to specify the details of the generated image, such as the figure posture, the overall layout, etc., which is equivalent to raising the upper limit of the AIGC.
However, for many ordinary users that Taobao Tmall wants to serve, the understanding and use cost of these guidance methods is still very high.
In August, 2003, a new technology, IP-Adapter, appeared, which lowered the threshold of generating images on demand stably again.
IP-Adapter is equivalent to opening up a new method of "using images as prompts" and can also be used as a guiding condition in ControlNet.
In this way, e-commerce sellers can make good use of ready-made product photos in the process of drawing pictures, without learning to master prompt skills or other professional drawing knowledge, which is equivalent to further reducing the lower limit of AIGC.
The basic practical problems are solved, and the deeper needs are highlighted.
Stable Diffusion belongs to the pre-training model, and the understanding of the world mainly comes from the data in the training stage.
The field of e-commerce is changing rapidly, and a large number of new products are available every day, which requires AI to be constantly updated and continuously learned.
At this time, LoRA, a fast fine-tuning technology from the big language model, comes in handy, and it is used as a method of "knowledge/concept injection" in the field of AIGC.
Specifically, LoRA will freeze most of the weights of the model and update only a small part when fine-tuning. At the same time, the updated weights can be separated separately, each of which is only tens to hundreds of MB in size.
In the practice of Taobao Tmall, LoRA is equivalent to building a digital avatar for goods and models, and depositing digital assets for merchants, which can further generate more diverse pictures of goods or models.
After adding LoRA to the process, the problem of poor controllability of generation appears again. How to make the model generated by LoRA strike a balance between image and beauty, and how to accurately restore the details of the products generated by LoRA are all application challenges that need to be further solved.
In addition, Taobao Tmall is still exploring a brand-new knowledge injection technology that can use commodity concepts for image generation without training, and can provide commodity images directly in the reasoning stage, which greatly reduces the application cost of knowledge injection technology and improves real-time.
AIGC needs to be applied on a large scale, and there are so many enhancements and transformations around the image generation model Stable Diffusion itself.
However, in the actual workflow, there is still a lot of work to be done in order to greatly reduce the complexity of drawing operation.
For example, in order to meet the needs of the e-commerce field that the commodity subject must be accurately presented, SAM segmentation model is added, which separates the commodity subject first and then adapts to the environmental background generated by AI.
In this way, it ensures the accurate presentation of the main body of the goods, avoids the appearance of "the goods are not in the right version", and can make the main body of the goods seamlessly integrate with the background, so that the light and shadow look flawless and have a sense of placing things.
Next, we should combine local detail repair, super resolution, adding filters and other links to improve the image quality and reach the commercial level.
With the application of AIGC, the process of large-scale application of large-scale models in e-commerce field is equally difficult.
In particular, what Taobao Tmall wants to do is not a pure professional model, but a general model with strong professional field ability to analyze the natural language problems of Taobao users’ long tails and understand more accurate user intentions.
To achieve this, we must first increase the professional knowledge of e-commerce on the basis of the general model and train in the data of e-commerce industry accumulated for many years.
But at this time, the general big model is still writing text according to the input. For example, when the user asks a question, the model is likely to supplement several similar questions according to the format, instead of answering this question.
The solution to this problem in the industry, AI, is aligned with human preferences. SFT (Instruction Fine-tuning) allows the large model to learn how to complete the user’s instructions, while RLHF (Human Feedback Reinforcement Learning) allows the large model to learn what answers meet human preferences. In the practice of Taobao Tmall, users’ feedback preferences can be continuously iterated after the product is launched.
The next thing to be solved is the "illusion problem" in the big model answer. In this regard, the Taobao Tmall technical team solved the problem from both inside and outside the model.
A large number of e-commerce industry data are introduced into the model, that is, in the model training stage.
Outside the model, through RAG technology, different knowledge bases are called for different problems to obtain real-time updated commodity information.
Furthermore, in order to solve the problem that external tools need to be called in real time, Taobao Tmall technical team uses Tool learning technology to optimize the ability of large model to understand tools, select tools and call tools, and provide an interpretable tool calling path, which makes the answers more accurate and richer.
For example, an ordinary query made by users in Taobao Ask not only calls the ability to generate large models, but also calls the ability to recommend Taobao products and the ability to recommend videos in the content community. More complex scenes, such as travel planning, will also call the partner Feizhu’s machine wine reservation ability.
Finally, a series of large model products, represented by Taobao Ask, have the ability of intention recognition, task planning, memory and using external tools on the basis of large models, and are developing in the direction of Agent, which is the next stage that the large model industry will compete for.
What is the AI score in the Double Eleven exam?
In the past, double eleven every year was a big test for technology and engineering. This year, for the addition of AI, it has added a taste of the first actual combat test.
Now, does this wave of AI achievements of Taobao Tmall Group also represent some clear new trends?
First of all, AI technology innovation has expanded the boundaries of e-commerce and increased the possibilities.
Shopping never needs to know exactly what to buy and search. It has become that as long as there is demand, you can ask AI.
Even if you don’t know what to buy before or don’t think you can solve the demand by buying goods at all, it may be recommended by AI for you.
Secondly, AI technology innovation has improved the working methods and production efficiency of businesses.
Similar to the buyer’s situation, but different, the seller is recommended by AI for background functions and business tools.
In the past, merchants used the Qianniu workbench in the background of Taobao Tmall merchants, which had a low learning cost and too many functions. Many of them didn’t know where it was or even the existence of a certain function.
Under the blessing of AI, it has also become possible to find AI if there is any demand in the store operation process, let AI locate the appropriate function, or answer the merchant’s questions through RAG technology.
Finally, although the Double Eleven has passed, a lot of interaction with AI has left valuable experience and data.
Taobao Tmall Group has a complete e-commerce industry data and industry experience, and by virtue of it, it has created large-scale AI products. During the Double Eleven period, it generated a large number of user feedback data, which can be re-invested in algorithm improvement, and finally formed a snowball effect.
OpenAI Developer Day marks the growing volume of the AI industry. Paul Graham, founder of YC, suggested that if he is not eliminated in the competition, he needs to do the following:
Not only rely on AI, but also rely on deep domain knowledge in specific fields.Establish a very close relationship with end users.
These two points happen to be what Taobao Tmall Group is naturally good at.
There is a saying in the AIGC industry, "One day in AI, one year in the world". I look forward to what new shopping experience AI can bring us next year.
Reporting/feedback