但可能打不過公園裡的老大爺？

巴黎奧運(yùn)會正在如火如荼地進(jìn)行中，乒乓球項(xiàng)目備受關(guān)注。同時，機(jī)器人打乒乓球也取得了新突破。

剛剛，DeepMind 提出了第一個在競技乒乓球比賽中達(dá)到人類業(yè)餘選手水平的學(xué)習(xí)型機(jī)器人智能體。

論文地址：https://arxiv.org/pdf/2408.03906

DeepMind 這個機(jī)器人打乒乓球什麼程度呢？大概和人類業(yè)餘選手不相上下：

正手反手都會：

對手採用多種打法，機(jī)器人也能招架得?。?/p>

不過，比賽激烈程度似乎不如公園老闆對戰(zhàn)。

對機(jī)器人來說，乒乓球運(yùn)動需要掌握複雜的低階技能和策略性玩法，需要長期訓(xùn)練。 DeepMind 認(rèn)為戰(zhàn)略上次優(yōu)但可以熟練地執(zhí)行低階技能可能是更好的選擇。這使乒乓球與國際象棋、圍棋等純粹的戰(zhàn)略遊戲區(qū)分開來。

因此，乒乓球是提升機(jī)器人能力的一個有價值的基準(zhǔn)，包括高速運(yùn)動、即時精確和戰(zhàn)略決策、系統(tǒng)設(shè)計(jì)以及與人類對手直接競爭。

對於這一點(diǎn)，Google DeepMind 首席科學(xué)家稱讚道：「乒乓球機(jī)器人將有助於我們解決高速控制和感知問題?！?p>

該研究進(jìn)行了29 場機(jī)器人與人類的乒乓球比賽，其中機(jī)器人獲勝45% (13/29)。所有人類選手都是機(jī)器人未見過的玩家，從初學(xué)者到錦標(biāo)賽選手能力不等。

雖然該機(jī)器人輸?shù)袅怂信c最高級別玩家的比賽，但它贏得了100% 的與初學(xué)者的比賽，在與中級選手的對戰(zhàn)中贏得了55% 的比賽，展現(xiàn)出人類業(yè)餘選手的水平。

總的來說，研究的貢獻(xiàn)包括：

1. 提出一個分層和模組化的策略架構(gòu)，其中包括：

2. 低階控制器及其詳細(xì)的技能描述器，這些描述器對智能體的能力進(jìn)行建模並有助於彌合模擬與真實(shí)的差距；

3. 選擇低階技能的高階控制器。

4. 實(shí)現(xiàn)零樣本模擬到真實(shí)的技術(shù)，包括定義基於現(xiàn)實(shí)世界的任務(wù)分佈的迭代方法，並定義自動課程（automatic curriculum）。

5. 即時適應(yīng)未見過的對手。

方法介紹

該智能體由一個低階技能庫和一個高階控制器組成。低階技能庫專注於乒乓球的某個特定方面，例如正手上旋球、反手瞄準(zhǔn)或正手發(fā)球。除了包含訓(xùn)練策略，研究還在線上下和線上收集和儲存有關(guān)每個低階技能的優(yōu)勢、劣勢和限制的資訊。而負(fù)責(zé)協(xié)調(diào)低階技能的高階控制器會根據(jù)當(dāng)前遊戲統(tǒng)計(jì)、技能描述選擇最佳技能。

此外，該研究還收集了少量的人類和人類對打的比賽數(shù)據(jù)，作為初始任務(wù)條件的種子，數(shù)據(jù)集包括位置、速度和旋轉(zhuǎn)的資訊。然後使用強(qiáng)化學(xué)習(xí)在模擬環(huán)境中訓(xùn)練智能體， 並採用一些現(xiàn)有技術(shù)，將策略無縫部署到真實(shí)硬體中。?

該智能體與人類一起對打以產(chǎn)生更多訓(xùn)練數(shù)據(jù)，隨著機(jī)器人的持續(xù)學(xué)習(xí)，遊戲標(biāo)準(zhǔn)變得越來越複雜，以此讓智能體學(xué)習(xí)越來越複雜的動作。這種混合的「模擬 - 現(xiàn)實(shí)」循環(huán)創(chuàng)建了一個自動教學(xué)，使機(jī)器人的技能隨著時間的推移而不斷提高。?

Layered control

Layered control mainly includes the following parts:

• Table tennis playing style: The high-level controller (HLC, high-level controller) first decides which playing style to use (forehand or Backhand);

• Adjustment: Maintain each HLC's preference (H value) online based on statistics from matches against opponents;

• Select the most effective skill: HLC pairs shortlisted players based on adjusted H value Sampling by LLC;

• Updates: H-values ??and opponent statistics are updated until the end of the game.

Results

The researchers compared the agent with 29 table tennis players of different levels, including beginners, intermediate, advanced and advanced + skills. Human players played three games against the robot according to standard table tennis rules, but the rules were slightly modified because the robot was unable to serve.

Facing all opponents, the robot won 45% of matches and 46% of games. Broken down by skill level, the bot won all its matches against beginners, lost all its matches against Advanced and Advanced+ players, and won 55% of its matches against Intermediate players. This shows that the agent reaches the level of an intermediate human player in table tennis rounds.

The reason why robots cannot beat advanced players is due to physical and technical limitations, including reaction speed, camera sensing capabilities, rotation processing, etc., which are difficult to accurately model in a simulation environment.

Sparring with robots is also very attractive

Research participants said that they enjoyed playing with robots very much and gave the robots high ratings in terms of "interesting" and "attractive" . They also unanimously expressed that they were "very willing" to fight the robot again. During free time, they played with the robot for an average of 4 minutes and 06 seconds over 5 minutes.

The robot is not good at backspin

The participant with the best skills mentioned that the robot is not good at handling backspin. To test this observation, the researchers plotted the robot's landing rate against the ball's spin, and the results showed that the robot's landing rate dropped significantly as it faced more backspin balls. This flaw is partly caused by the robot trying to avoid colliding with the table when handling low balls, and secondly by the fact that it is really difficult to determine the ball's spin in real time.

^{Reference link:}

^{https://sites.google.com/view/competitive-robot-table-tennis/home?utm_source&utm_medium&utm_campaign&utm_content&pli=1}

以上是DeepMind機(jī)器人打乒乓球，正手、反手溜到飛起，全勝人類初學(xué)者的詳細(xì)內(nèi)容。更多資訊請關(guān)注PHP中文網(wǎng)其他相關(guān)文章！