This article introduces how to efficiently view Hadoop logs in Debian system. Hadoop's log aggregation function centrally stores the logs of each node for easy management. The following steps will guide you on how to view these logs:
Step 1: Make sure log aggregation is enabled
First, confirm that log aggregation is enabled for all nodes in the Hadoop cluster. This is usually configured in yarn-site.xml
configuration file:
<property> <name>yarn.log-aggregation-enable</name> <value>true</value> </property> <property> <name>yarn.log-aggregation.retain-seconds</name> <value>172800</value> </property> <property> <name>yarn.nodemanager.remote-app-log-dir</name> <value>/tmp/logs</value> </property> <property> <name>yarn.nodemanager.remote-app-log-dir-suffix</name> <value>logs</value> </property> <property> <name>yarn.log.server.url</name> <value>http://hadoop-node0:19888/jobhistory/logs</value> </property>
Step 2: Access logs through the Web UI
After log aggregation, you can view the logs through Hadoop's ResourceManager Web UI or the Job History Server Web UI:
- ResourceManager Web UI: The default address is
http://<resourcemanager-hostname> :8088/cluster/apps/FINISHED</resourcemanager-hostname>
. - Job History Server Web UI: The default address is
http://<jobhistory-server-hostname> :19888/jobhistory</jobhistory-server-hostname>
.
Step 3: Use the command line tool to view the log
If you need to read the logs directly from HDFS, you can use the hdfs dfs
command:
hdfs dfs -cat /path/to/logs/*
Step 4: Use a text editor to view the log
You can open and view log files using any text editor (such as nano
, vim
):
sudo nano /path/to/logs/application-<application-id> -container-<container-id> .log</container-id></application-id>
Note that log file paths and URLs may vary depending on Hadoop cluster configuration. Please refer to your Hadoop cluster configuration file for the exact path.
The above is the detailed content of How to view Debian Hadoop logs. For more information, please follow other related articles on the PHP Chinese website!

Hot AI Tools

Undress AI Tool
Undress images for free

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Chinese version
Chinese version, very easy to use

Zend Studio 13.0.1
Powerful PHP integrated development environment

Dreamweaver CS6
Visual web development tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

Hot Topics









This article has selected several top Python "finished" project websites and high-level "blockbuster" learning resource portals for you. Whether you are looking for development inspiration, observing and learning master-level source code, or systematically improving your practical capabilities, these platforms are not to be missed and can help you grow into a Python master quickly.

Blockchain confirmation time refers to the time it takes for a transaction to be broadcasted to be packaged by a block and written to the chain. The confirmation speeds of different chains vary. 1. Bitcoin produces blocks on average in 10 minutes, and it is recommended to confirm 6 times to ensure security; 2. Ethereum produces blocks in about 12 seconds, and 1-3 times can be confirmed, and most transactions are completed within 1 minute; 3. The BSC chain block time is about 3 seconds, suitable for high-frequency trading; 4. The TRON tide block time is 1-3 seconds, suitable for real-time transfer; 5. The Polygon block time is about 2 seconds, with low fees, and is widely used in DeFi and NFT. Trading hash (TxID) is required for query and confirmation status. Recommended platforms include: 1. Ouyi OKX, which supports multi-chain transaction query; 2. Binance, suitable for BSC chain; 3. Huobi HT

Blockchain browser is a must-have on-chain query tool for Web3 users. 1. It serves as a "search engine" in the decentralized world, allowing users to openly and transparently verify all records on the blockchain; 2. The core functions include querying transaction details, viewing account information, exploring block data and tracking smart contracts; 3. When tracking transactions, you need to obtain the transaction hash, select the browser corresponding to the public chain, and enter the hash to view the status, address, amount and fee details; 4. Confirm whether the transaction is successful through the browser is a key step to ensure the security of digital assets. Proficient use can help users better understand and participate in the blockchain ecosystem, thereby operating more safely and stably in the decentralized world.

This article has selected a series of top-level finished product resource websites for Vue developers and learners. Through these platforms, you can browse, learn, and even reuse massive high-quality Vue complete projects online for free, thereby quickly improving your development skills and project practice capabilities.

To collect user behavior data, you need to record browsing, search, purchase and other information into the database through PHP, and clean and analyze it to explore interest preferences; 2. The selection of recommendation algorithms should be determined based on data characteristics: based on content, collaborative filtering, rules or mixed recommendations; 3. Collaborative filtering can be implemented in PHP to calculate user cosine similarity, select K nearest neighbors, weighted prediction scores and recommend high-scoring products; 4. Performance evaluation uses accuracy, recall, F1 value and CTR, conversion rate and verify the effect through A/B tests; 5. Cold start problems can be alleviated through product attributes, user registration information, popular recommendations and expert evaluations; 6. Performance optimization methods include cached recommendation results, asynchronous processing, distributed computing and SQL query optimization, thereby improving recommendation efficiency and user experience.

The era of compliant crypto trading in Hong Kong has begun, and six potential currencies have attracted attention: 1. Bitcoin (BTC) is a must-have asset as digital gold; 2. Ethereum (ETH) has become the second largest compliance option with its strong ecosystem; 3. Ripple (XRP) has compliance potential due to its cross-border payment application and liquidity advantages; 4. ADA (ADA) meets regulatory preferences with its academic foundation and stability; 5. Litecoin (LTC) is regarded as a security option due to its long history and stable community; 6. Binance Coin (BNB) is likely to be selected based on its high market value and wide ecosystem. Investors can first trade these assets through mainstream platforms such as Binance, Ouyi, Huobi, and Gate.io to prepare for the future opening of the Hong Kong retail market.

Digital currency is a decentralized electronic currency based on blockchain technology. Its value is determined by market supply and demand. It is globally universal, transparent in transactions and has certain anonymity. 1. Binance is the largest platform in the world with transaction volume, providing rich currencies and a strong ecosystem. 2. OkX has a comprehensive product line, supporting derivatives trading and Web3 accounts, suitable for advanced users; 3. Huobi (Huobi/HTX) has a long history, is safe and stable, and is deeply trusted by old users; 4. Gat

In 2025, the cryptocurrency market will be driven by three major factors: clear policy supervision, in-depth institutional participation and technological innovation. 1. The United States may introduce a comprehensive crypto bill, and global regulatory coordination (such as the EU MiCA) will enhance the legitimacy of the industry; 2. Institutional funds will enter large-scale through compliant products such as Bitcoin and Ethereum ETFs, and exchanges such as Binance, OKX, Huobi, Gate.io and Coinbase will serve as core infrastructure to undertake institutional traffic; 3. The popularization of Layer 2 network will promote the scale of applications, and the tokenization of real-world assets (RWA) is expected to inject trillion-dollar liquidity into DeFi. The integration of AI and crypto will give birth to a new paradigm, jointly driving the growth of the industry's intrinsic value.
