Internet No Longer Open


Guest Mores
 Share

Recommended Posts

Guest Mores

I'd heard a rumor recently.  And I don't know of any way to verify it.  I'm wondering if anyone can help me.  What I heard was somewhat reminiscent of some things said about net neutrality.

Apparently, there are only a few internet platforms that run and/or have a heavy handed control of about 98% of the websites out there.  A person wanted to go for a month without Amazon, Google, Facebook, and a few other big names.  He found it was almost impossible.

EXAMPLE: These platforms start out with giving out free accounts.  But when you want to go to an unrelated site, many of them want you to sign up for their site using a Google account or a Facebook account.  And if you don't have them, a growing number of sites don't give you an option of signing up for a site specific account.  So, unless you're part of one or more of these platforms, you're severely limited in how much of the web you can browse.

This was just one of the most minor devices that was purportedly used.  Another thing is that Amazon (so goes the rumor) actually controls a HUGE number of websites.  While people tend to believe they run their own site, the site services are available because of some kind of background web that all websites are painted on.  Amazon owns most of that background web.

I know nothing of the inner workings of the internet.  I just know what I see when I browse.  How would this be verified?

THE REASON I ASK is that @MormonGator had recently posted the standard libertarian free-market idea that any private entity has the right to control their own stuff.  This is true while there is an open market.  But through government intervention, it seems that these few groups have created a communications oligarchy protected by the government, preventing free entry by would be competitors.  Absent such competition, the free-market does not function.

Edited by Mores
Link to comment
Share on other sites

13 minutes ago, Mores said:

I'd heard a rumor recently.  And I don't know of any way to verify it.  I'm wondering if anyone can help me.  What I heard was somewhat reminiscent of some things said about net neutrality.

Apparently, there are only a few internet platforms that run and/or have a heavy handed control of about 98% of the websites out there.  A person wanted to go for a month without Amazon, Google, Facebook, and a few other big names.  He found it was almost impossible.  

That 98% number is not true and not even close to it.  I'll give you some points to ponder because it's almost impossible to know exactly how big is the internet, so giving any percentage would be, at best, a potshot in the dark.

So, the size of the internet depends on how many servers are tied to it.  When the internet first came online back in 1969, it was compromised of 4 computers.  Today, who knows how many there are... there's no organization that has ever attempted to index all of them.  It used to be that the US Government owned the assignment of URIs (the internet phone book) but it has been ceded to a non-government international organization ICANN.  At least, ICANN knows how many addresses are there in the phone book.  But the number of addresses doesn't tell you how many computers or how much data is tied to each address.

So, here's one stat that I like to shock people with (hah!)... Google is the most popular search engine... Back in 2015, they have claimed to have indexed over 200 terrabytes of data for their search engine.  But Google estimates this to be ONLY 0.004% of all the data available on the internet at the time.

How is Google's 0.004% claim possible?  Okay, here's an example - I'm a programmer for a multi-national company.  All of my company's data is on the internet but not accessible by Google.  Here's another example... the Dark Web is ginormous.   It is not accessible through Google either and not accessible to normies.  You get to it through IP-masking software like TOR.  It's a world much different than Google-able web and is getting larger.

So people may wonder... so Dark Web stuff is not accessed by normies, so why is it relevant to the conversation?  This is because, the Boomers, Gen Xers, and the older Millenials who comprise most of the normies are getting older and...  more and more GenZs and younger (or aptly called the iGen) are the ones who think "normie web" is like us going to a payphone - it's too normie.  Let's put it this way - if your iGen kid has an active social media account on Facebook, he'd be like the Duranie among the Nirvana fans back in the 90's.  So these iGen kids hang out in the Dark Web unbeknownst to their "what is TOR?" parents who are just happy to buy them the latest gadget.  So, it is within the realm of possibility that "normie web" will die off to be replaced by a changing internet culture of more "control-averse" paradigms.  I predict this control-averse paradigm will get larger because even with 4chan (the gateway drug to the Dark Web) you see people now abandoning it as too "normie" because it became too controlling.  But yeah, that's just one possibility and just my opinion on the matter.

 

13 minutes ago, Mores said:

EXAMPLE: These platforms start out with giving out free accounts.  But when you want to go to an unrelated site, many of them want you to sign up for their site using a Google account or a Facebook account.  And if you don't have them, a growing number of sites don't give you an option of signing up for a site specific account.  So, unless you're part of one or more of these platforms, you're severely limited in how much of the web you can browse.  

Okay, Google is owned by Alphabet.  Alphabet owns a bunch of stuff like Waymo (self-driving cars), Verily (Healthcare database), etc.  Google itself owns a bunch of companies like YouTube, Android, Chrome, Cloud, etc. etc.  So, if you're within this universe of Alphabet-owned stuff, chances are your Alphabet/Google account will have universal access to them.

Of course, this does not limit you in how much of the web you can browse.  It simply tells you... you're not in Alphabet/Google land anymore so your Alphabet/Google account won't work there.

 

13 minutes ago, Mores said:

This was just one of the most minor devices that was purportedly used.  Another thing is that Amazon (so goes the rumor) actually controls a HUGE number of websites.  While people tend to believe they run their own site, the site services are available because of some kind of background web that all websites are painted on.  Amazon owns most of that background web.  

Amazon is another Tech Giant.  But it is a different company than Google with different customer services.  So, Google owns Cloud (data server).  Amazon has its own cloud computing company called CloudEndure.  Also, Amazon provides web hosting services like Google does, so you can upload your website onto Amazon's servers if you don't want to buy your own.

About "background web".  Google has more of these services than Amazon.  Amazon's engine is mainly for online business building - that's webpage hosting, site traffic analytics, buy/sell analytics, etc. etc.  Oh, and Amazon also bought out an Israeli chip manufacturing company.  Google, on the other hand, sells more varied "engine" stuff like their proprietary search engine algorithm, mapping services, etc. to other companies.

 

13 minutes ago, Mores said:

I know nothing of the inner workings of the internet.  I just know what I see when I browse.  How would this be verified?

You can't.  The internet is truly the Wild Wild West.

Link to comment
Share on other sites

I'll see if I can help clarify a bit.  I'm a professional web developer so I have some knowledge here.

57 minutes ago, Mores said:

EXAMPLE: These platforms start out with giving out free accounts.  But when you want to go to an unrelated site, many of them want you to sign up for their site using a Google account or a Facebook account.  And if you don't have them, a growing number of sites don't give you an option of signing up for a site specific account.  So, unless you're part of one or more of these platforms, you're severely limited in how much of the web you can browse.

It sounds like what this is talking about is SSO (Single Sign On Authentication).  Essentially it's a mechanism where websites outsource their user authentication.  In other words, you sign in using your Google or FB account to prove who you are.  This is pretty common and I've actually built a SSO mechanism into one of the web applications I built, and Google was one of the options.  It doesn't necessarily mean the site is controlled or influenced by Google, FB or whatever.  There are other SSO providers though they're usually limited to a specific cluster of sites (called a Federation).  For example, for the CVRG project, Johns Hopkins, U Chicago, Emory and others had a federated authenticator where logging in using your credentials form any of these universities would get you into the system.  (And what a pain that was to build.  I had to implement several different mechanisms before they settled on one they liked.)  

That said, I don't know of any examples I've seen where the site doesn't have an alternate way to log in, at least not yet.

57 minutes ago, Mores said:

This was just one of the most minor devices that was purportedly used.  Another thing is that Amazon (so goes the rumor) actually controls a HUGE number of websites.  While people tend to believe they run their own site, the site services are available because of some kind of background web that all websites are painted on.  Amazon owns most of that background web.

 Sort of.  The way it works is that Amazon, among others, owns cloud and web hosting services.  What that  means is that if you wanted to have a website but didn't have the equipment to host it yourself, you can have it hosted by a webhosting service.  Your website lives on their servers and you can access and set it up remotely.  Amazon is one of the larger webhosting services and also have their own industry standard cloud... which is basically just webhosting with a bunch of features and software products already built in.

Amazon also owns some of the physical infrastructure on the Internet, but that's mostly for their own use.  They don't have any control beyond that.

57 minutes ago, Mores said:

THE REASON I ASK is that @MormonGator had recently posted the standard libertarian free-market idea that any private entity has the right to control their own stuff.  This is true while there is an open market.  But through government intervention, it seems that these few groups have created a communications oligarchy protected by the government, preventing free entry by would be competitors.  Absent such competition, the free-market does not function.

Most of the Internet infrastructure is controlled by a variety of different entities, including universities, private corporations, governments and the military.  No one entity controls it, even within the USA.

Link to comment
Share on other sites

1 hour ago, Mores said:

THE REASON I ASK is that @MormonGator had recently posted the standard libertarian free-market idea that any private entity has the right to control their own stuff.  This is true while there is an open market.  But through government intervention, it seems that these few groups have created a communications oligarchy protected by the government, preventing free entry by would be competitors.  Absent such competition, the free-market does not function.

The problem here is not "the internet" per se.  Rather, it is the nature of the monopoly.  Social Media is, by its nature, dependent on location where social interactions conglomerate.  Free-market principles like competition do not apply to this issue because... you can't just come up with your own location and continue to engage in the social interaction.  It's like the kid who wanted to engage in conversation with the cafeteria kids who won't let him sit at their table can't just join the conversation by declaring his table as the new location for conversation.  Make sense? 

There is no good way to address this issue.  The government MUST stay out of the internet.  I am leary about Trump taking on this issue because usually, when a government gets involved, regulations on the internet will follow and we don't want that.  I am completely against government-imposed Net Neutrality.

 

 

Edited by anatess2
Link to comment
Share on other sites

Guest Mores
38 minutes ago, unixknight said:

I'll see if I can help clarify a bit.  I'm a professional web developer so I have some knowledge here.

Thank you, I appreciate it.

38 minutes ago, unixknight said:

It sounds like what this is talking about is SSO (Single Sign On Authentication).  Essentially it's a mechanism where websites outsource their user authentication.  In other words, you sign in using your Google or FB account to prove who you are.  This is pretty common and I've actually built a SSO mechanism into one of the web applications I built, and Google was one of the options.  It doesn't necessarily mean the site is controlled or influenced by Google, FB or whatever.  There are other SSO providers though they're usually limited to a specific cluster of sites (called a Federation).  For example, for the CVRG project, Johns Hopkins, U Chicago, Emory and others had a federated authenticator where logging in using your credentials form any of these universities would get you into the system.  (And what a pain that was to build.  I had to implement several different mechanisms before they settled on one they liked.)  

That said, I don't know of any examples I've seen where the site doesn't have an alternate way to log in, at least not yet.

I have come across a few.  But like I said, this was only one mechanism of MANY that create the dearth of access.

38 minutes ago, unixknight said:

 Sort of.  The way it works is that Amazon, among others, owns cloud and web hosting services.  What that  means is that if you wanted to have a website but didn't have the equipment to host it yourself, you can have it hosted by a webhosting service. 

Yes, that sounds like what I meant by the "background".  But I don't have the web developer vocabulary to describe what it was.  Thank you.

38 minutes ago, unixknight said:

Your website lives on their servers and you can access and set it up remotely.  Amazon is one of the larger webhosting services and also have their own industry standard cloud... which is basically just webhosting with a bunch of features and software products already built in.

This sounds more like what I was talking about.  So, I was told that most of the regularly available websites to "normal people" are all owned by a very small number of companies.  Is that not so?

38 minutes ago, unixknight said:

Amazon also owns some of the physical infrastructure on the Internet, but that's mostly for their own use.  They don't have any control beyond that.

They don't control the websites they host?  They don't have the ability to shut them off or re-route traffic?

38 minutes ago, unixknight said:

Most of the Internet infrastructure is controlled by a variety of different entities, including universities, private corporations, governments and the military.  No one entity controls it, even within the USA.

Perhaps I misunderstand this aspect of it.  But it seemed that apart from governments, the other entities depend on web hosting services to even connect their internal infrastructure to the public infrastructure.  Is that not so?

Link to comment
Share on other sites

Guest Mores
27 minutes ago, anatess2 said:

you can't just come up with your own location and continue to engage in the social interaction. 

There's the problem that I was talking about.  Anyone SHOULD be able to simply create their own social media site and advertise with the right places to allow alternative avenues of free speech.  But it seems such avenues are limited.

How did Third Hour get set up?  How much is actually shared here?  It seems that niche sites like this one are "allowed" by TPTB because such sites are A) Small & B) Absent any high profile individuals.

Do you know of any social media site that has open access to political debate that has at least 1,000,000 active participants that can hear words from high profile individuals?  Last I heard Jordan Peterson and a couple of his friends recently made an announcement that they are leaving Patreon because he was feeling the beginnings of biased nudging from them.

How easy would it be to set up such a website that was at least OPEN to all?  Then if it got a big enough profile, what mechnisms are in place to prevent them from being shut down or taken over by larger companies that are exercising monopolistic power over them?

Link to comment
Share on other sites

27 minutes ago, Mores said:

There's the problem that I was talking about.  Anyone SHOULD be able to simply create their own social media site and advertise with the right places to allow alternative avenues of free speech.  But it seems such avenues are limited.  

This is missing the objective of the social interaction.  

So, okay, probably best to make it into a scenario.  So, Twitter decides to ostracize conservative voices.  So, you think, ok, we'll just create our own version of Twitter to give a platform to conservative voices.  Well, Twitter users who are engaged in socio-political discussion who don't like conservative voices are not going to go to Twitter-C.  And non-conservative content creators are not going to have incentives to market in Twitter-C.  So, what you end up with is Twitter and the segregated site Twitter-C (which will forever be known as the Conservative site) - and the two do not interact.  So, it defeats the purpose of the interactions of Social Media.  Now, add to that the fact that the POTUS uses Twitter... Twitter-C will be the lunch table over there by the garbage cans.

 

Quote

How did Third Hour get set up?  How much is actually shared here?  It seems that niche sites like this one are "allowed" by TPTB because such sites are A) Small & B) Absent any high profile individuals.

3H is not an open social media.  It has a stated purpose - promotion of LDS beliefs.  It has no reason to open the platform to engage with those dissident to the LDS belief system.  So it's apples and oranges.

 

Quote

Do you know of any social media site that has open access to political debate that has at least 1,000,000 active participants that can hear words from high profile individuals?  Last I heard Jordan Peterson and a couple of his friends recently made an announcement that they are leaving Patreon because he was feeling the beginnings of biased nudging from them.

Patreon is not a social media platform.  It is a payment processing platform.  JBP and Dave Rubin dumped Patreon and is trying to create their own payment processing platform to give social media content creators an alternative payment processing system.  They hit a brick wall when they realized Patreon is not the only hurdle... the banks themselves can block payment to content creators.

JBP and Dave Rubin use the main social media content platforms - YouTube, Twitter, et. al. - for their social media content creations and also leverage the dark web to interact with their followers.

 

Quote

How easy would it be to set up such a website that was at least OPEN to all?  Then if it got a big enough profile, what mechnisms are in place to prevent them from being shut down or taken over by larger companies that are exercising monopolistic power over them?

It's very easy to have your own table in the cafeteria.  If you're not beholden to anybody and you stay within legal grounds where your server resides, you can't get shut down.  But, it is not as easy for that table to become desegregated from the main table.  You'd basically be shouting at the main table from your own corner without getting heard or responded to by the main table.  Which is exactly the problem with segregation.  If you manage to become the Main Table great!  That means the old main table - Twitter, in our example - died, or they get reduced to the table shouting at the Main Table and nobody hearing or responding.  But the journey there will be like the end of that movie The One with Jet Li. 

Edited by anatess2
Link to comment
Share on other sites

20 minutes ago, Mores said:

This sounds more like what I was talking about.  So, I was told that most of the regularly available websites to "normal people" are all owned by a very small number of companies.  Is that not so?

I don't know what the proportion is of corporate owned sites to private sites, but I suspect there are a ton more privately owned sites.  The thing to keep in mind, and I'll use my own websites as examples, is that we have to understand what, exactly, a website is and what we actually own.

First, you need a place for the site to live on.  That's a server.  You might own the server, or you might pay for hosting.   I have two websites and I take a different approach for each.  On my gaming club website, I own the server, which is a Dell computer in my basement.  I installed all the software and configured it to run my site.  The computer is physically located in my house so I own 100% of my site's infrastructure.  I pay nobody for hosting it and I can take it down, modify it, update it, etc. whenever I feel like it.  That site can't handle very much traffic, but since my gaming club is only a handful of people it's perfectly fine.  I also have a separate computer which is a Minecraft server used by my club.  Same idea, just does a different job.

My sci fi review site is different.  I had the idea of a much larger possible volume of traffic and the physical equipment to handle a very large load is expensive.  So, I rent space on somebody else's servers.  There's a hosting service run by a friend who does Christian websites, so I know my site is hosted by honorable people who host other honorable sites.  I pay a yearly fee for not only the space for that site to live, but also to have them maintain the software and hardware infrastructure that the site runs on.  I have direct control over the content of my site, but no direct control over the hardware or software installed on the server.

The second thing to consider is the URL (Universal Resource Locator).  Every website has an address, just like every building has an address.  That address is a hard to remember string of numbers and decimals so to make it easier to navigate the web,  you can "buy" a URL which is the name you want your site to have (like thirdhour.org)  So when you type 'thirdhour.org' into your web browser, the browser automatically consults a URL library where it gets the actual IP address of the site.  We never see it, the browser just shows thirdhour.org, but we actually are hitting the IP address.  To get a URL of your own for your site, you go to one of the various companies that serve as brokers of URLs, and request what you want.  So if you wanted a site called 'thewisdomofmorse.com' you'd go to one of these companies and request it.  They verify that it isn't already in  use somewhere else, and if it's not, you pay a yearly fee to 'own' the URL.  You'd then use that URL to point to the address of your website.  I say 'own' in quotes because if you don't pay the yearly fee, you lose ownership of it and it becomes available for someone else.

20 minutes ago, Mores said:

They don't control the websites they host?  They don't have the ability to shut them off or re-route traffic?

  They control the physical hardware and software, but typically there would be contractual issues if they were to tamper with traffic to the site.  They can physically shut them down, but they'd need a legally justifiable reason to do so.  As they say, it's all in the fine print.  In my example above, my sci fi review site could be shut down by the webhosting service if I were to violate the terms of use.  My gaming club site is, on the other hand, entirely under my own control.  As long as I break no laws, I can put absolutely whatever I want on it.

20 minutes ago, Mores said:

Perhaps I misunderstand this aspect of it.  But it seemed that apart from governments, the other entities depend on web hosting services to even connect their internal infrastructure to the public infrastructure.  Is that not so?

Yes.  My gaming club site is connected to the world through my Verizon FiOS service.  Verizon has no control over the content of my site, though if I were to  host something illegal then they can easily provide my IP address and my physical home address to law enforcement.  In theory, they could monitor the content that passes through their infrastructure and possibly throttle my connection speed or shut me down.  That's part of the debate over Net Neutrality... to what extent should such services be able to do that?  As of now, they aren't generally known to abuse that ability, so it's a think to keep an eye on. 

Link to comment
Share on other sites

Okay just to avoid confusion, @Mores.  @unixknight and I (after my first post) are actually talking about 2 different aspects of the internet - @unixknight is addressing general websites.  I'm addressing Social Media platforms.  Web servers, in non-computing speak, is like your house.  You can buy your own house, own your own property, or you can rent someone else's house.  The concept of property ownership is the same between a house and a web server.  Social Media, on the other hand, is the same as the "Public Square".  You can declare your house as the Public Square where people gather to interact with each other.  But having many Public Squares segregates people because there's only limited time in a day so people can't go to different houses saying the same things.  So, they either choose the most popular ones to interact with the most number of people on any topic, or they go to niche squares that only discuss specific topics of interest (like thirdhour).

The censorship issue is only a problem in Open-Topic Social Media platforms (houses declared as Public Squares).  The problem is not applicable to Web servers (non-Public Square Houses) or in niche Social Media platforms (censored to the niche by design).

 

 

Edited by anatess2
Link to comment
Share on other sites

4 minutes ago, anatess2 said:

The censorship issue is only a problem in Open-Topic Social Media platforms (Public Squares).  The problem is not applicable to Web servers (Houses) or in niche Social Media platforms (censored to the niche by design).

True, though the potential for ISPs (Internet Service Providers) to start blocking sites for other than purely legal reasons does exist, which would extend the censorship problem beyond social media.

Mind you, I still oppose the idea of net neutrality because things are a lot more complex than that...

Link to comment
Share on other sites

2 minutes ago, unixknight said:

True, though the potential for ISPs (Internet Service Providers) to start blocking sites for other than purely legal reasons does exist, which would extend the censorship problem beyond social media.

Mind you, I still oppose the idea of net neutrality because things are a lot more complex than that...

In the case of ISP's... this is operable under Free Market competition... except... because of the old problem with utility companies in bed with local government, it poses a challenge with the throttling of competition by decades-old local regulations.  For example - Google Fiber has a hard time penetrating AT&T and Comcast areas because... there are local government regulations preventing Google to lay physical hardware down existing utility tunnels that AT&T and Comcast already have lines in.

And the air is controlled by FCC... 

Link to comment
Share on other sites

Guest Scott
2 hours ago, anatess2 said:

So, here's one stat that I like to shock people with (hah!)... Google is the most popular search engine... Back in 2015, they have claimed to have indexed over 200 terrabytes of data for their search engine. 

But Google estimates this to be ONLY 0.004% of all the data available on the internet at the time.

Is this really right?  Our relatively small engineering firm has that much indexed.

I would think that Google has a lot more than that indexed for their search engine.

 

Link to comment
Share on other sites

1 minute ago, Scott said:

Is this really right?  Our relatively small engineering firm has that much indexed.

I would think that Google has a lot more than that indexed for their search engine.

 

The thing to keep in mind is that Google doesn't index the Dark Web, which is the vast body of the iceberg to the tip which is the Clearnet (The Internet post people know, and is searchable by search engines.)  This website is on the Clearnet.  If you want to access the Dark Web, you need special browser software.

 

Link to comment
Share on other sites

Guest Mores
1 hour ago, unixknight said:

I don't know what the proportion is of corporate owned sites to private sites, but I suspect there are a ton more privately owned sites.  The thing to keep in mind, and I'll use my own websites as examples, is that we have to understand what, exactly, a website is and what we actually own.

I want to emphasize again that I'm talking about the ownership of the web hosting that the websites are on, not just the websites themselves.

1 hour ago, unixknight said:

They control the physical hardware and software, but typically there would be contractual issues if they were to tamper with traffic to the site.  They can physically shut them down, but they'd need a legally justifiable reason to do so.  As they say, it's all in the fine print.  In my example above, my sci fi review site could be shut down by the webhosting service if I were to violate the terms of use.  My gaming club site is, on the other hand, entirely under my own control.  As long as I break no laws, I can put absolutely whatever I want on it.

Yes.  My gaming club site is connected to the world through my Verizon FiOS service.  Verizon has no control over the content of my site, though if I were to  host something illegal then they can easily provide my IP address and my physical home address to law enforcement.  In theory, they could monitor the content that passes through their infrastructure and possibly throttle my connection speed or shut me down.  That's part of the debate over Net Neutrality... to what extent should such services be able to do that?  As of now, they aren't generally known to abuse that ability, so it's a think to keep an eye on. 

This is closer to answering my question.  It makes sense that there would be contractual issues with fettering access to their customers' sites.  But how many hosting services are there?

Link to comment
Share on other sites

4 minutes ago, Scott said:

Is this really right?  Our relatively small engineering firm has that much indexed.

I would think that Google has a lot more than that indexed for their search engine.

 

From a 2015 interview with the former CEO of Google... somebody Schmidt.  That's over 3 years ago.  Probably twice that by now.

The interesting thing is... the searchable internet is currently estimated as still less than 1% of the internet.  The deep web is like the part of the iceberg underneath the water.

Link to comment
Share on other sites

Just now, Mores said:

I want to emphasize again that I'm talking about the ownership of the web hosting that the websites are on, not just the websites themselves.

This is closer to answering my question.  It makes sense that there would be contractual issues with fettering access to their customers' sites.  But how many hosting services are there?

Gotcha.  I don't know, though my friend's webhosting company is fairly small.  Anybody with the ability to invest the capital to buy some good hardware and a solid, reliable Internet connection can start their own.  I don't think she even has much of a staff, just her husband and herself as far as I know.

Link to comment
Share on other sites

Guest Scott
5 minutes ago, anatess2 said:

From a 2015 interview with the former CEO of Google... somebody Schmidt.  That's over 3 years ago.  Probably twice that by now.

The interesting thing is... the searchable internet is currently estimated as still less than 1% of the internet.  The deep web is like the part of the iceberg underneath the water.

I found some of the info you are referring to:

https://www.seeker.com/how-much-of-the-internet-is-hidden-1792697912.html

Still, even when counting only the searchable internet, I'm still surprised that it is only 200 terabytes that is indexed for Googles search engine.  I would have thought it would be a lot more than that.

Link to comment
Share on other sites

15 minutes ago, Mores said:

I want to emphasize again that I'm talking about the ownership of the web hosting that the websites are on, not just the websites themselves.

This is closer to answering my question.  It makes sense that there would be contractual issues with fettering access to their customers' sites.  But how many hosting services are there?

 

13 minutes ago, unixknight said:

Gotcha.  I don't know, though my friend's webhosting company is fairly small.  Anybody with the ability to invest the capital to buy some good hardware and a solid, reliable Internet connection can start their own.  I don't think she even has much of a staff, just her husband and herself as far as I know.

There are a slew of hosting services, big and small, free or expensive, all over the world.  Also, all ISPs offer free hosting services.  They have the traffic capacity so most people who don't have their own hardware will simply use their ISPs hardware.  But this is irrelevant to the issue of "open internet" because you can move your site to any hosting service and people accessing the site won't even have to know about it.  So, you can't close the internet in this manner.  This is is actually how people get around legalities.  For example, child porn is illegal in the US.  So child porn peddlers go to other countries and have their websites hosted there and it's still accessible in the US like any other US sites.

 

Edited by anatess2
Link to comment
Share on other sites

Guest Mores
13 minutes ago, Scott said:

I found some of the info you are referring to:

https://www.seeker.com/how-much-of-the-internet-is-hidden-1792697912.html

Still, even when counting only the searchable internet, I'm still surprised that it is only 200 terabytes that is indexed for Googles search engine.  I would have thought it would be a lot more than that.

I don't know if that is a correct number or not.  I really don't know.  But I'd say that 200 terabytes of "indexing" could account for a large percentage of sites out there.  This doesn't mean that everything on all web pages is stored on a Google server.  That's not what it's saying at all.  I certainly hope it's not.  I'd certainly have a hard time believing that.

Link to comment
Share on other sites

10 minutes ago, Scott said:

I found some of the info you are referring to:

https://www.seeker.com/how-much-of-the-internet-is-hidden-1792697912.html

Still, even when counting only the searchable internet, I'm still surprised that it is only 200 terabytes that is indexed for Googles search engine.  I would have thought it would be a lot more than that.

Another interesting tidbit you might be interested in... so, also back in 2015... the WABAC Machine project claimed to have archived 23 petabytes of data spanning 20 years... my brain can't mentally deal with that number of zeros and I haven't really paid attention to how WABAC crawls sites for archive... so... if you know this info, you might be able to guesstimate how much data is sitting around searchable web on any given day... at least from back in 2015.

Link to comment
Share on other sites

8 minutes ago, Mores said:

I don't know if that is a correct number or not.  I really don't know.  But I'd say that 200 terabytes of "indexing" could account for a large percentage of sites out there.  This doesn't mean that everything on all web pages is stored on a Google server.  That's not what it's saying at all.  I certainly hope it's not.  I'd certainly have a hard time believing that.

It's the estimate by the former CEO of Google from 2015.  And it's not "a large percentage of sites out there" - it is simply the sum total of the searchable internet (Google being the largest search engine, it would be safe to logically assume that if it's searchable, it's indexed by Google).  This is only estimated to be less then 1% of all the "sites out there".

Link to comment
Share on other sites

Guest Scott
19 hours ago, anatess2 said:

It's the estimate by the former CEO of Google from 2015.  

Interestingly, it seems that the same figures the CEO quoted in 2015 were also quoted long before that.   

Here they are from 2010:

https://www.webanalyticsworld.net/2010/11/google-indexes-only-0004-of-all-data-on.html

Here is the same figure from 2008:

https://www.quora.com/How-much-disk-space-is-required-to-operate-a-search-engine-the-size-of-Google-Yahoo-Bing-or-DuckDuckGo-and-how-much-would-it-cost

Link to comment
Share on other sites

23 minutes ago, Scott said:

Interestingly, it seems that the same figures the CEO quoted in 2015 were also quoted long before that.   

Here they are from 2010:

https://www.webanalyticsworld.net/2010/11/google-indexes-only-0004-of-all-data-on.html

Here is the same figure from 2008:

https://www.quora.com/How-much-disk-space-is-required-to-operate-a-search-engine-the-size-of-Google-Yahoo-Bing-or-DuckDuckGo-and-how-much-would-it-cost

Whoa mama!

See... that's the thing about it, isn't it?  There's really zero possible way of knowing how large is the internet and how much of it is not accessible by normies.  It's TRULY the Wild Wild West and I like it that way.  ;)

Man... I kinda miss the days when we would hack into the school's computer with our rotary phones.  Hah hah.

Link to comment
Share on other sites

Guest Mores
52 minutes ago, Scott said:

Interestingly, it seems that the same figures the CEO quoted in 2015 were also quoted long before that.   

Here they are from 2010:

https://www.webanalyticsworld.net/2010/11/google-indexes-only-0004-of-all-data-on.html

Here is the same figure from 2008:

https://www.quora.com/How-much-disk-space-is-required-to-operate-a-search-engine-the-size-of-Google-Yahoo-Bing-or-DuckDuckGo-and-how-much-would-it-cost

Well, that's fine and all... but the fact is that most people don't get around the dark web much.  Even if the vast majority of the web is free and open, how many people even know how to get there?

Your graphic mentions that there are over 5 exabytes on the internet.  Where did that come from?  Not that I doubt it.  But I just wonder what it includes.  And how did they come up with that 0.0004% (or whatever) number?

My understanding from previous verbiage was that Google has 200 TB of indexing data of the 100s of petabytes or possibly the 5 exabytes of actual data on the regular web.

Yet, even all those (approaching) literally astronomical numbers doesn't get to the heart of the question I'm asking.  The SEARCH ENGINE may have only indexed so much of the searchable links.  But Google the CORPORATION has so many more arms and branches.  It is essentially a supersized technological conglomerate as are these other corporations that I mentioned.  We only see their very public face.  But their partners and subsidiaries are not necessarily well known.

I'm really wondering just how powerful they really are, and how much power they really have over our ability to communicate on the internet.  If it is to the point where they can censor significantly (which we've seen happen a LOT) then we need to ensure that there is free entry into the market, or else we have the robber baron situation --  a government created problem which made a government solution popular at the cost of more of our freedoms.

Yes, we can say "At least we have the dark web."  And maybe that is the solution.  I don't know since I've only just heard about this myself from this thread.  So, I'm obviously far from an expert on it.  Could this easily become a solution to online censorship?

BTW, (not criticizing, just observing) I don't believe I've really heard anything here that says this rumor is unfounded or untrue.  Just throwing around a lot of numbers as if they mean something.  Well, I know a bit about numbers (see what I did there, "bit" :) ).   I just don't see anything here that tells me it is wrong.  I don't see anything that says it is right.  But there are other blips in current events that tells me it may very well be so.

Edited by Mores
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
 Share