Question for This Episode

Where did today's Large Language Models (LLMs) learn all that vast knowledge? The answer lies in the traces we've unintentionally left on the web for the past 30 years. The "World Wide Web (WWW)," born in the 1990s, was more than just a means of communication; it was the process of writing the world's largest "AI textbook" in human history.

Core Connection: From History to the Present

If the operating systems and networks covered in the previous episodes created the "way for computers to talk," the World Wide Web created the standard for the "container to hold information." Tim Berners-Lee's proposal of HTML and HTTP wove fragmented human knowledge into one giant web.

The reason this "connection" is important is because of the explosive accumulation and standardization of data. As text, images, and videos began to be digitized and piled onto the web, the era of "Big Data," where artificial intelligence could finally learn, was opened. Without the web, the intelligence we use today, like ChatGPT or Claude, would not exist.

3 Decisive Moments of the Web that Opened the AI Era

1. HTML: Structuring and Labeling Knowledge

Unlike simple text files, HTML gave a hierarchy to information through tags like headings (h1), body text (p), and links (a). This structured data became a decisive hint for later AI crawlers to understand what information is important and how they are connected to each other.

2. Evolution of Search Engines and Indexing

The development of search engines from Yahoo to Google sophisticated the algorithms for finding "valuable information" among the vast data on the web. Google's "PageRank" algorithm quantified the correlation between data, which is in line with the "Attention" mechanism of modern AI.

3. Web 2.0 and User-Generated Content (UGC)

The emergence of blogs, Wikipedia, and social media brought daily human conversations and knowledge beyond information produced by a few experts onto the web. Thanks to this, AI was able to learn not only rigid encyclopedic knowledge but also human emotions, humor, and colloquial expressions.

Data Lessons to Remember in Practice

"Structure" comes before the quantity of data. One HTML tag, one piece of metadata changes the accuracy with which AI understands information.
Connected information is powerful. Information woven with links to other documents has a higher importance within the AI model than a single document.
The value of public data is eternal. The public materials we upload to the web today become nourishment for making even more powerful AI in the future.

Executive Execution Summary

Item	Execution Criteria
Content Strategy	Apply structural markup (Semantic Web) that AI can easily read.
Data Assetization	Manage internal data by refining it into web standard specifications.
Search Engine Optimization (SEO)	Optimize in a form that is easy to reference not only by search engines but also by "AI chatbots."
Ethics and Security	Establish security policies recognizing that public web data can be used for AI learning.
Success Signal	Increased frequency of company content being cited as a source of answers for major AI models.

Frequently Asked Questions (FAQ)

Q1. Does AI automatically get smarter as more web data becomes available?

Quality is as important as the quantity of data. Recently, "data contamination," where low-quality data or AI-generated data on the web is reused for AI training, is a serious topic of discussion.

Intranets also operate based on web technology. You need to organize internal documents according to web standards to save a lot of time and money when introducing a "private AI" later.

Q3. What will the next episode cover?

If data has been collected through the web, we will now cover the birth of cloud computing and distributed systems, the "containers" for processing that data. We'll look at how AI has come to think simultaneously on tens of thousands of servers beyond just one computer.

BigData Posts

The Path to AI 04: World Wide Web and the Democratization of Information, from Collective Intelligence to Artificial Intelligence