The best thing, by a long way, that Google Scholar has achieved is denying Elsevier & co a monopoly on academic search.
In most universities here in New Zealand, articles have to be published in a journal indexed by Elsevier's Scopus. Not in a Scopus-indexed journal, it does not count anymore than a reddit comment. This gives Elsevier tremendous power. But in CS/ML/AI most academics and students turn to Google Scholar first when doing searches.
I'm a proud user of sci-hub but when I was still in academics, I have never used it. My school has access to all the journals I ever needed, plus more old non-digitized ones I can borrow from library (including interlibrary access).
Nothing lasts forever, but the model of buying a paper for 40$ from Elsevier isn't much better.
Depending on the field there are other sources, but still a hit rate is about 85-90%.
sci-hub proper hasn't been updated since it's indefinite pause in december 2020.
Alternatives are of variable success depending on field. It might be better for CS/Math, but medicine and life sciences it's pretty bad.
scihub is dying unfortunately :( the good news is it is happening just as all the fields i'm interested in except for some experimental physics & biology have moved to OA
Google Scholar is extremely valuable to the academic community. I am afraid that Google will decide to scrap it someday, and we will be left with a number of inferior alternatives.
Fun fact about Google Scholar: it’s "free", but it’s just another soulless Google product - no clear strategy, no support, and a fragile proprietary dependency in what should be an open ecosystem. This creates inherent risks for the academic community. We need the equivalent of arXiv for Google Scholar
I miss the Google of yesteryear which had an altruistic streak and felt that enriching the world's ability to share and process information would ultimately accrue benefit to Google as well.
The Google of today is far more boring and less helpful.
For people unfamiliar, Semantic Scholar is run by the Allen Institute and has been researching accurate AI summarization and semantic search for years. Also they have support for author name changes.
It advertises itself as "from all fields of science" -- does that includes fields like economics? Sociology? Political science? What about law journals? In other words, is the coverage as broad? And if it doesn't include certain fields, where is the "science" line drawn?
And I'm curious if people find it to be as useful (or more) just in terms of UX, features, etc.
They are substantially smaller in coverage, but have higher quality in my experience. Remarkably, they are also willing to correct their data if you notify them. This of course in is stark contrast to Google Scholar where the metadata of papers is frequently wildly inaccurate. On top of this, Semantic Scholar shares their underlying data (although you need to request an API key). Overall, they have been growing slowly and steadily over the years and I have a lot of respect for what their team is doing for researchers such as myself.
Now for the less great.
They are pushing the concept of "Highly Influential Citations" [1] as their default metric, which to the best of my knowledge is based on a singular workshop publication that produced a classifier trained on about 500 training samples to classify citations. I am a very harsh critic of any metrics for scientific impact. But this is just utter madness. Guaranteeing that this metric is not grossly misleading is nearly impossible and it feels like the only reason they picked it is because Etzioni (AI2 head) is the last author of the workshop paper. It should have been at best a novelty metric and certainly not the default one.
Recently, they introduced their Semantic Reader functionality and are now pushing it as a default way to access PDFs on the website. Forcing you to click on a drop down to access plain PDFs. It may or may not be a great tool, but it feels somewhat obvious that they are attempting to use shady patterns to push you in the direction they want.
Lastly, they have started using Google Analytics. Which is not great, but I can understand why they go for the industry default.
Overall, I use them nearly daily and they are the best offering out there for my area of research. Although, I at times feel tempted to grab the data and create an alternative (simpler) frontend with fewer distractions and "modern" web nonsense.
Semantic Scholar's search is pretty good, but there are also a variety of other (paid) projects that expand on its API. Look at tools like Scite and LitMaps for what's possible with the semantic scholar dataset.
As for coverage, I think it focuses more on the life sciences, but I'm not positive about that.
Yes. On one hand I’d like Google to improve things a bit. There are some rough edges, which is a shame because it indexes some things that are not in Scopus or Web of Knowledge, like theses and preprint repositories. On the other hand I worry that some manager somewhere would kill it if they realised that it is still around.
I’m fairly sure they only exist because Larry/Sergei might give half a fuck if they killed it outright, and it has a small enough team that the cost savings for killing aren’t enough for Ruth to want to make that argument.
Every 1-2 months when Chrome updates I get banned by their throttling mechanism because I their extension makes too many requests and they see "unusual traffic"
It can take 1-2 weeks to go away and be able to use it. There's no way to get in contact with anyone. Tried the Chrome extension email, support forums.
It's a good reality check. There's no real support behind it and it can go away just like Google Reader did.
I think the motivations behind it are laudable, but they should not be the answer to the actual problem.
I have similar feeing for Gmail (it's effective anti spam engine), google maps and google docs (which pioneered shared docs. It feels outdated on many fronts now, but it was a pioneer).
As much as I try to "de-google" myself and try to avoid being trapped in the Google eco system, I'd definitely choose it over MS Office. I am stuck in the MS Office eco system at work. Some of their products are starting to improve in MS Office, but you can still tell it's a lot of hacks ontop of old systems. Especially when it comes to the whole teams/onedrive/sharepoint side of things.
One of my biggest gripes right now is that we heavily rely on Microsoft Teams. A lot of our work laptops still are stuck on 8gb of ram. I find Microsoft Teams can easily suck back a full gig or more or ram, especially when in a video call. From my understanding, Teams is running essentially like an Electron app (except using an Edge browser packaged).
I have no problem with web based apps, but man, some optimization is called for.
I use a decade-old NUC with plenty of RAM as a daily driver. It doesn't struggle with anything except MS Teams. It can churn through Zoom or Meet calls while compiling code. Teams is a bloated mess that makes the fans spin at max RPM.
It's crazy I can boot a kernel, with an entire graphics and network stack, X and a terminal in less than 200 MB but then the Teams webapp uses a massive amount of resources and grinds everything else to a halt.
Word 365 also becomes incredibly laggy on long documents with tons of comments, whereas Google Docs is just fine. But, apparently, this is also a thing on modern hardware. I guess these days Microsoft has little attention to detail.
It's funny because sometimes Teams uses more resources than the Edge browser. Despite Teams being Edge based for their application.
I think overall many companies have gotten lazy/sloppy when it comes to optimization. Game dev is even worse for this. I like how Microsoft products integrate with each other, but often the whole thing feels sloppy and unoptimized.
Good for users of Gmail, but is it a net good? Gmail spam prevention is great for the Google Apps orgs I manage. However, for the other inboxes the vast majority of spam they receive comes from @gmail.com
> Gmail spam prevention is great for the Google Apps orgs I manage.
Gmail is unlikely to let spam through.
But that doesn't make its spam filter great; it's also very prone to blocking personal communication on the grounds that it must actually have been spam. The principle of gmail's spam filter is just "don't let anything through".
It would be much better to get more spam and also not have my actual communications disappear.
It's probably not. You can put any domain you want on the "from" address. Just because it says it was from Gmail doesn't mean it actually was, unless it's signed with DKIM etc.
I had a domain for a while that people got spam "from" all the time. It had nothing to do with me and there was nothing I could do about it.
I run mail servers for myself, a couple of side projects, and some friends and family. A double-digit percentage of all spam caught by my filters is from Google's mail servers, not just forged @gmail.com addresses.
Of the "too big to block outright" spam senders, behind Twilio Sendgrid and Weebly, Google is currently #3. Amazon is a close #4. None of the top four currently have useful abuse reporting mechanisms... Sendgrid used to be OK, but they no longer seem to take any action. Google doesn't even accept abuse reports, which is ironic because "does not accept or act upon abuse reports" is criteria for being blocked by Google.
Most spam from Google is fake invoices and 419 scams. This is trivially filtered on my end, which makes it perplexing Google doesn't choose to do so. I can guarantee that exactly 0% of Gmail users sending out renewal invoices for "N0rton Anti-Virus" are legitimate.
The Spam I get from "gmail" and ends up in my spam folder is spoofed. The Spam I get from gmail and ends up in my inbox is from gmail. Spammers will mass-create accounts and mass-sell them to spammers.
anti-spam is only an issue if people dump their email anywhere.
I usually register my mail on webpages as first.last+webpage@mail.com and once they would spam this mail, it gets blacklisted.
I literally get only 1-3 real spam mails per month without any filter.
I used to require a "+..." on all emails. Any email that didn't have the "+..." was sent to Spam automagically. My family were whitelisted. I gave up, because too many websites (early on) refused to take the "+..." marker, so I ended up losing too much to Spam. It's easier to just let Google sort it out.
Not everyone's cup of tea, but quite nice if one can afford it: I have my personal domain and a catch-all inbox. So if I want to register at acme-co.xyz I will just use acmecoxyz@my-domain.tld
Maybe I should start using random words though? Wonder if someone will go bananas seeing their brand's name on my domain.
Yeah, I've had to explain that a couple times already, usually when dealing with customer support or in-person registrations.
And a "malicious" actor can get away with pretending to be another company by spoofing the username if they know your domain works like that. I don't think this has reached spammers' repertoire yet, but I wouldn't be surprised.
Eventually I'd like to have a way of generating random email addresses that accept mail on demand, and put everything else in quaraintine automatically.
I see this recommendation everywhere and I am genuinely surprised that it works. Any spammer can find out your real address since there is an obvious mapping from + addresses to your real address. An actual solution would hide this mapping.
Yeah. Fastmail masked addresses are random. The best you can do is guess that an address might be masked, due to it not being johnsmith@fastmail.com, but it provides no information about your real email address.
Google maps would only be a net good if the data was available under a free licence. As it is they take data from people that should have gone to a public project like OpenStreetMap.
At one point I contributed quite a bit to google maps, because it was the primary map system I was using at the time. Had I been using an OSM-based system, I would have made contributions there instead.
I ran into trouble because Open Topo does not report a stream the 7.5" series does. There's serious data quality issues that can make it not work for some applications.
> 18. A paw-sitive contribution to Physics. F.D.C Willard (otherwise known as Chester, the Siamese cat) is listed as a co-author on an article entitled: “Two, Three, and Four-Atom Exchange Effects” that explores the magnetic properties of solid helium-3 and how interactions between its atoms influence its behavior at extremely low temperatures. Chester’s starring role came about because his co-author/owner, Jack H. Hetherington wrote the entire paper with the plural “we” instead of a single “I.”
---
'Two-, Three-, and Four-Atom Exchange Effects in bcc 3He' by J. H. Hetherington and F. D. C. Willard [0, 1, 2]
Sir Andre Geim [0], the only person in the world who received both the real Nobel prize in Physics and the Ig Nobel prize co-authored one of his articles [1] with his hamster Tisha.
Time goes by fast. It's interesting to think how authors son is now 20 as well.
Another interesting thing is little popup form at the end of post asking me if my opinion of Google changed for the better after reading the post. I mean maybe a bit, b the form definitely knocked the score back down.
I'd go to the card catalog (index), turn my question into a bag of words (tokenize), fetch all the cards matching each token (posting lists), drop cards which didn't include enough of the tokens (posting list intersection), ordering the cards by the number of tokens they matched (keyword match ranking), filter at some cutoff, and then reorder based on the h-index of the author (page rank). Then I would read each paper in order, following citations in a breadth-first manner.
(the above is a joke comparing old school library work to search engines circa 2000; I didn't actually do all those steps. I'd usually just find the most recent review article and read the papers it cited).
21. Google Scholar will deny access to you if you (need to) self-host a VPN on a common VPS provider. Being a Google product, it also can’t be special-cased in your routing table. (I genuinely had to retrain myself to use Google Scholar again once I no longer had that need.)
22. Switching on sort by date will impose a filter to papers published within the year, and you cannot do anything about that.
There is filter by date and sort by date. The former works. The latter, when enabled, even adds a banner on top of the page (in large but gray type) that says “Articles added in the last year, sorted by date”, and resets any filter you might have set before.
If it ever returned time-sorted results without limit, that was long in the past. It has truncated results to one year for the last several years I have used Scholar.
It seems so intentionally "broken", I can only guess it is to prevent scraping? Since searching for generic-ish search terms and sorting by date is a common scraping strategy.
Still, you'd think they'd do a cutoff of e.g. 500 or 1,000 items rather than filter by the past year.
So I can't help but wonder if it's a contractual limitation insisted on by publishers? Since the publishers also don't want all their papers being spidered via Scholar? It feels kind of like a limitation a lawyer came up with.
Google employs a lot of people from academia. Scholar is used and loved by a lot of people within Google. It's been around for two decades. I really don't think it's going anywhere.
Reader was used and loved by a LOT of people WITHIN google, but it was shut down (and the leadership that loved it even made arguments in front of the company why it "had to be shut down").
AFAICT Scholar remains because Anurag built up massive cred in the early years (he was a critically important search engineer) with Larry Page and kept his infra costs and headcount really small, while also taking advantage of search infra).
Reader usage was declining because the application was not being developed. The other reason they mentioned is that it would have required a lot of work to rewrite the app to be consistent with the new user data policies being put into place.
I'd not known about "F.D.C. Willard" — the nom de plume of a Michigan State physics professor's Siamese cat, Chester — who was listed as a co-author of a number of the professor's physics papers.
I did not know about PDF Scholar Readee extension [1]. Unfortunately the reason is that I use Firefox only (and safari iOS) and it is not available there. The AI outlines will be useful and I can think of myself using it.
I do not want to comment on number 20. I really wished that I joined CERN 10 years earlier but then it is the mistake of my parents :)
> Would he want to continue working on Scholar for another ten years? “One always believes there are other opportunities, but the problem is how to pursue them when you are in a place you like and you have been doing really well. I can do problems that seem very interesting me — but the biggest impact I can possible make is helping people who are solving the world’s problems to be more efficient. If I can make the world’s researchers ten percent more efficient, consider the cumulative impact of that. So if I ended up spending the next ten years going this, I think I would be extremely happy.”
Has he still been working on it in the 10 years since this article? His name is in the byline of the new blog post, but it's not clear from that how much he's been working on it.
12-13 years ago, I ran the system that inlined Scholar and other results on the main search result pages. Anurag was still involved, but AFAIR Alex, the other author of the post who also had been there from the start, worked on most code changes. I would guess that things are more or less the same today. (Because it had such limited headcount, Scholar was known to lag behind other services when it came to code/infrastructure migrations.)
Thanks for that inside scoop, even if it's a bit dated; I wonder if they read this discussion, perhaps.
An important feature request would be a view where only peer-reviewed publications (specifically, not ArXiv and other pre-print archives) are included in the citation counts, and self-citations are also excluded.
A way to download all citation sources would also be a great nice-to-have.
A reminder to everyone: if you want a "legal" copy of a paper you can always just try emailing one of the first authors. They will 99.99% send you back a PDF.
I wish GScholar wouldn't embrace bibliometrics so much. Sort papers by date (most recent papers first) by default on an author's page rather than by citation count, or at least give author the choice to individually opt-in to sort by date by default.
Google Scholar is fantastic stuff. I am so grateful for it. It’s crazy how easy it is to find papers these days by just going to it. University library search functions are completely useless in comparison.
Is this a jokey reference to that time Paul Graham upset large amounts of Nigerians on Twitter? Or, rather, genuine concern at the thought that the article may have been generated by chatbots?
I've been using Google Scholar for a long time, but I'm finding ChatGPT search with well-crafted prompts gets more focused and relevant results than a complex keyword search on GS does. However it's often still easier to find a link to the pdf version of the paper using GS, but then scihub is still an option and can work when all else fails.
Huh. I tried the "Listen to article" button, because I knew it was going to be generated and was curious to hear how it sounded.
Interestingly, it highlighted the words as it read. I haven't seen that before online. Not sure how useful it is (especially for anyone interested in this particular topic), but I thought it was a neat innovation nevertheless.
The best thing, by a long way, that Google Scholar has achieved is denying Elsevier & co a monopoly on academic search.
In most universities here in New Zealand, articles have to be published in a journal indexed by Elsevier's Scopus. Not in a Scopus-indexed journal, it does not count anymore than a reddit comment. This gives Elsevier tremendous power. But in CS/ML/AI most academics and students turn to Google Scholar first when doing searches.
or turn to sci-hub and annas-arhive :)
I'm a proud user of sci-hub but when I was still in academics, I have never used it. My school has access to all the journals I ever needed, plus more old non-digitized ones I can borrow from library (including interlibrary access).
You use Google Scholar to find papers you're interested in, then use sci-hub to actually read them.
indeed... and use Zotero with the correct plugin to download them automagically
sci-hub hasn't been updated in 4 years and the sources for annas-archive like nexus-stc are seriously hit or miss (depends on the field).
Nothing lasts forever, but the model of buying a paper for 40$ from Elsevier isn't much better. Depending on the field there are other sources, but still a hit rate is about 85-90%.
Does sci-hub have up to date content these days?
Having pretty wide journal access through my institution means I don’t need to reach out to sci-hub.
sci-hub proper hasn't been updated since it's indefinite pause in december 2020. Alternatives are of variable success depending on field. It might be better for CS/Math, but medicine and life sciences it's pretty bad.
i believe they paused due to an indian court injunction and the case was heard this year, does anyone know any update?
scihub is dying unfortunately :( the good news is it is happening just as all the fields i'm interested in except for some experimental physics & biology have moved to OA
oa resources have really kicked it into high gear post covid. They used to be kind of a joke, but they're actually competitive now. It's nice to see.
I believe NIH's directive that all intramural and extramural research must be published OA has helped move things in that direction quite a lot.
Google Scholar is extremely valuable to the academic community. I am afraid that Google will decide to scrap it someday, and we will be left with a number of inferior alternatives.
Like annas archive?
Fun fact about Google Scholar: it’s "free", but it’s just another soulless Google product - no clear strategy, no support, and a fragile proprietary dependency in what should be an open ecosystem. This creates inherent risks for the academic community. We need the equivalent of arXiv for Google Scholar
I miss the Google of yesteryear which had an altruistic streak and felt that enriching the world's ability to share and process information would ultimately accrue benefit to Google as well.
The Google of today is far more boring and less helpful.
And that is semantic scholar, https://www.semanticscholar.org/
For people unfamiliar, Semantic Scholar is run by the Allen Institute and has been researching accurate AI summarization and semantic search for years. Also they have support for author name changes.
How does it compare with Google Scholar?
It advertises itself as "from all fields of science" -- does that includes fields like economics? Sociology? Political science? What about law journals? In other words, is the coverage as broad? And if it doesn't include certain fields, where is the "science" line drawn?
And I'm curious if people find it to be as useful (or more) just in terms of UX, features, etc.
They are substantially smaller in coverage, but have higher quality in my experience. Remarkably, they are also willing to correct their data if you notify them. This of course in is stark contrast to Google Scholar where the metadata of papers is frequently wildly inaccurate. On top of this, Semantic Scholar shares their underlying data (although you need to request an API key). Overall, they have been growing slowly and steadily over the years and I have a lot of respect for what their team is doing for researchers such as myself.
Now for the less great.
They are pushing the concept of "Highly Influential Citations" [1] as their default metric, which to the best of my knowledge is based on a singular workshop publication that produced a classifier trained on about 500 training samples to classify citations. I am a very harsh critic of any metrics for scientific impact. But this is just utter madness. Guaranteeing that this metric is not grossly misleading is nearly impossible and it feels like the only reason they picked it is because Etzioni (AI2 head) is the last author of the workshop paper. It should have been at best a novelty metric and certainly not the default one.
[1]: https://webflow.semanticscholar.org/faq/influential-citation...
Recently, they introduced their Semantic Reader functionality and are now pushing it as a default way to access PDFs on the website. Forcing you to click on a drop down to access plain PDFs. It may or may not be a great tool, but it feels somewhat obvious that they are attempting to use shady patterns to push you in the direction they want.
Lastly, they have started using Google Analytics. Which is not great, but I can understand why they go for the industry default.
Overall, I use them nearly daily and they are the best offering out there for my area of research. Although, I at times feel tempted to grab the data and create an alternative (simpler) frontend with fewer distractions and "modern" web nonsense.
Semantic Scholar's search is pretty good, but there are also a variety of other (paid) projects that expand on its API. Look at tools like Scite and LitMaps for what's possible with the semantic scholar dataset.
As for coverage, I think it focuses more on the life sciences, but I'm not positive about that.
OpenAlex is a really good here too, including their API. They’re also the inheritors of the Microsoft Academic Graph, fully open source and open data:
https://openalex.org
The Invest in Open site has a good directory of open tools.
https://infrafinder.investinopen.org/solutions
Yes. On one hand I’d like Google to improve things a bit. There are some rough edges, which is a shame because it indexes some things that are not in Scopus or Web of Knowledge, like theses and preprint repositories. On the other hand I worry that some manager somewhere would kill it if they realised that it is still around.
I’m fairly sure they only exist because Larry/Sergei might give half a fuck if they killed it outright, and it has a small enough team that the cost savings for killing aren’t enough for Ruth to want to make that argument.
Every 1-2 months when Chrome updates I get banned by their throttling mechanism because I their extension makes too many requests and they see "unusual traffic"
It can take 1-2 weeks to go away and be able to use it. There's no way to get in contact with anyone. Tried the Chrome extension email, support forums.
It's a good reality check. There's no real support behind it and it can go away just like Google Reader did.
I think the motivations behind it are laudable, but they should not be the answer to the actual problem.
Yes, Google deserves to be distrusted and avoided as a whole, but Google Scholar is a genuinely net good for humanity.
I have similar feeing for Gmail (it's effective anti spam engine), google maps and google docs (which pioneered shared docs. It feels outdated on many fronts now, but it was a pioneer).
Try MS OneDrive before calling google docs outdated
Google spanks everyone else on robustness and responsiveness
As much as I try to "de-google" myself and try to avoid being trapped in the Google eco system, I'd definitely choose it over MS Office. I am stuck in the MS Office eco system at work. Some of their products are starting to improve in MS Office, but you can still tell it's a lot of hacks ontop of old systems. Especially when it comes to the whole teams/onedrive/sharepoint side of things.
One of my biggest gripes right now is that we heavily rely on Microsoft Teams. A lot of our work laptops still are stuck on 8gb of ram. I find Microsoft Teams can easily suck back a full gig or more or ram, especially when in a video call. From my understanding, Teams is running essentially like an Electron app (except using an Edge browser packaged).
I have no problem with web based apps, but man, some optimization is called for.
I use a decade-old NUC with plenty of RAM as a daily driver. It doesn't struggle with anything except MS Teams. It can churn through Zoom or Meet calls while compiling code. Teams is a bloated mess that makes the fans spin at max RPM.
It's crazy I can boot a kernel, with an entire graphics and network stack, X and a terminal in less than 200 MB but then the Teams webapp uses a massive amount of resources and grinds everything else to a halt.
Word 365 also becomes incredibly laggy on long documents with tons of comments, whereas Google Docs is just fine. But, apparently, this is also a thing on modern hardware. I guess these days Microsoft has little attention to detail.
It's funny because sometimes Teams uses more resources than the Edge browser. Despite Teams being Edge based for their application.
I think overall many companies have gotten lazy/sloppy when it comes to optimization. Game dev is even worse for this. I like how Microsoft products integrate with each other, but often the whole thing feels sloppy and unoptimized.
Yes until it fails
https://www.theverge.com/2023/11/27/23978591/google-drive-de...
That issue got resolved in a few days [1] -- and for each and every one of these extremely rare events at Google, you'll find similar ones at MS.
I am referring to robustness at scale and every day: Google released auto-save years before MS. MS pales in comparison in the UX.
Note: I have no vested interest in Google, not ex-googler, etc.
[1] https://support.google.com/drive/thread/245861992/drive-for-...
Good for users of Gmail, but is it a net good? Gmail spam prevention is great for the Google Apps orgs I manage. However, for the other inboxes the vast majority of spam they receive comes from @gmail.com
> Gmail spam prevention is great for the Google Apps orgs I manage.
Gmail is unlikely to let spam through.
But that doesn't make its spam filter great; it's also very prone to blocking personal communication on the grounds that it must actually have been spam. The principle of gmail's spam filter is just "don't let anything through".
It would be much better to get more spam and also not have my actual communications disappear.
"Google is evil, except for all the Google products Google produced"
Honestly, if we compare Google to Amazon, Microsoft, Apple, and Meta, isn't Google the least evil one?
Most of the spam I get is from gmail. Maybe they should apply their so effective spam engine to outgoing mail as well...
It's probably not. You can put any domain you want on the "from" address. Just because it says it was from Gmail doesn't mean it actually was, unless it's signed with DKIM etc.
I had a domain for a while that people got spam "from" all the time. It had nothing to do with me and there was nothing I could do about it.
I run mail servers for myself, a couple of side projects, and some friends and family. A double-digit percentage of all spam caught by my filters is from Google's mail servers, not just forged @gmail.com addresses.
Of the "too big to block outright" spam senders, behind Twilio Sendgrid and Weebly, Google is currently #3. Amazon is a close #4. None of the top four currently have useful abuse reporting mechanisms... Sendgrid used to be OK, but they no longer seem to take any action. Google doesn't even accept abuse reports, which is ironic because "does not accept or act upon abuse reports" is criteria for being blocked by Google.
Most spam from Google is fake invoices and 419 scams. This is trivially filtered on my end, which makes it perplexing Google doesn't choose to do so. I can guarantee that exactly 0% of Gmail users sending out renewal invoices for "N0rton Anti-Virus" are legitimate.
The Spam I get from "gmail" and ends up in my spam folder is spoofed. The Spam I get from gmail and ends up in my inbox is from gmail. Spammers will mass-create accounts and mass-sell them to spammers.
I would hope google has DKIM and SPF set.
anti-spam is only an issue if people dump their email anywhere. I usually register my mail on webpages as first.last+webpage@mail.com and once they would spam this mail, it gets blacklisted.
I literally get only 1-3 real spam mails per month without any filter.
Words great, until a page rejects email with a '+' in it.
Or just knows about this Gmail trick (it's been 20 years already) and sends spam to your real mailbox.
Actually, I am surprised _any_ spammy website these days would even honor the part after the +, and not just directly send to the real mailbox name.
I used to require a "+..." on all emails. Any email that didn't have the "+..." was sent to Spam automagically. My family were whitelisted. I gave up, because too many websites (early on) refused to take the "+..." marker, so I ended up losing too much to Spam. It's easier to just let Google sort it out.
It's part of RFC 5233 Sieve Email Filtering: Subaddress Extension
Not everyone's cup of tea, but quite nice if one can afford it: I have my personal domain and a catch-all inbox. So if I want to register at acme-co.xyz I will just use acmecoxyz@my-domain.tld
Maybe I should start using random words though? Wonder if someone will go bananas seeing their brand's name on my domain.
Yeah, I've had to explain that a couple times already, usually when dealing with customer support or in-person registrations.
And a "malicious" actor can get away with pretending to be another company by spoofing the username if they know your domain works like that. I don't think this has reached spammers' repertoire yet, but I wouldn't be surprised.
Eventually I'd like to have a way of generating random email addresses that accept mail on demand, and put everything else in quaraintine automatically.
dots are ignored, can filter by john.doe@gmail.com
not sure about capital letters
I see this recommendation everywhere and I am genuinely surprised that it works. Any spammer can find out your real address since there is an obvious mapping from + addresses to your real address. An actual solution would hide this mapping.
Yeah. Fastmail masked addresses are random. The best you can do is guess that an address might be masked, due to it not being johnsmith@fastmail.com, but it provides no information about your real email address.
Google maps would only be a net good if the data was available under a free licence. As it is they take data from people that should have gone to a public project like OpenStreetMap.
"take", these people would never have produced any data if gmaps wasn't there...
At one point I contributed quite a bit to google maps, because it was the primary map system I was using at the time. Had I been using an OSM-based system, I would have made contributions there instead.
indeed, osm can't paint itself like a victim, it needs good end products to bring in contributors.
I ran into trouble because Open Topo does not report a stream the 7.5" series does. There's serious data quality issues that can make it not work for some applications.
> 18. A paw-sitive contribution to Physics. F.D.C Willard (otherwise known as Chester, the Siamese cat) is listed as a co-author on an article entitled: “Two, Three, and Four-Atom Exchange Effects” that explores the magnetic properties of solid helium-3 and how interactions between its atoms influence its behavior at extremely low temperatures. Chester’s starring role came about because his co-author/owner, Jack H. Hetherington wrote the entire paper with the plural “we” instead of a single “I.”
---
'Two-, Three-, and Four-Atom Exchange Effects in bcc 3He' by J. H. Hetherington and F. D. C. Willard [0, 1, 2]
[0] https://xkeys.com/media/wysiwyg/smartwave/porto/category/abo...
[1] https://xkeys.com/about/jackspages/fdcwillard.html
[2] https://en.wikipedia.org/wiki/F._D._C._Willard
Sir Andre Geim [0], the only person in the world who received both the real Nobel prize in Physics and the Ig Nobel prize co-authored one of his articles [1] with his hamster Tisha.
[0] https://en.wikipedia.org/wiki/Andre_Geim
[1] https://repository.ubn.ru.nl//bitstream/handle/2066/249681/2...
I use Google Scholar daily and it's been a fantastic resource. Google Scholar with Zotero completes my articles search and storage.
Btw, Anurag's last name is misspelt under the picture. It reads "Achurya" instead of "Acharya"
Edit: They fixed it
Time goes by fast. It's interesting to think how authors son is now 20 as well.
Another interesting thing is little popup form at the end of post asking me if my opinion of Google changed for the better after reading the post. I mean maybe a bit, b the form definitely knocked the score back down.
Google Scholar is so good. I started doing research right when it came out and it was amazingly helpful. I can’t imagine how it was done before.
I'd go to the card catalog (index), turn my question into a bag of words (tokenize), fetch all the cards matching each token (posting lists), drop cards which didn't include enough of the tokens (posting list intersection), ordering the cards by the number of tokens they matched (keyword match ranking), filter at some cutoff, and then reorder based on the h-index of the author (page rank). Then I would read each paper in order, following citations in a breadth-first manner.
(the above is a joke comparing old school library work to search engines circa 2000; I didn't actually do all those steps. I'd usually just find the most recent review article and read the papers it cited).
There are alternatives, like Web of Knowledge. You basically need to be in a Uni for that though.
I would go to the library and pull volumes of Science Citation Index off the shelves. Yes, Google Scholar was a revolution.
21. Google Scholar will deny access to you if you (need to) self-host a VPN on a common VPS provider. Being a Google product, it also can’t be special-cased in your routing table. (I genuinely had to retrain myself to use Google Scholar again once I no longer had that need.)
22. Switching on sort by date will impose a filter to papers published within the year, and you cannot do anything about that.
> 22. Switching on sort by date will impose a filter to papers published within the year, and you cannot do anything about that.
!!! And here I thought it's been broken for years, and a sign of decay due to lack of internal support.
I swear this was working for me until literally today, it was really useful to find older ML papers?!
There is filter by date and sort by date. The former works. The latter, when enabled, even adds a banner on top of the page (in large but gray type) that says “Articles added in the last year, sorted by date”, and resets any filter you might have set before.
Was this change ever logged or noted some way? Or did it just show up one day?
If it ever returned time-sorted results without limit, that was long in the past. It has truncated results to one year for the last several years I have used Scholar.
It seems so intentionally "broken", I can only guess it is to prevent scraping? Since searching for generic-ish search terms and sorting by date is a common scraping strategy.
Still, you'd think they'd do a cutoff of e.g. 500 or 1,000 items rather than filter by the past year.
So I can't help but wonder if it's a contractual limitation insisted on by publishers? Since the publishers also don't want all their papers being spidered via Scholar? It feels kind of like a limitation a lawyer came up with.
oh no
they remembered google scholar exists
it's a great product and I don't trust google at all not to break it or mess with it
Google employs a lot of people from academia. Scholar is used and loved by a lot of people within Google. It's been around for two decades. I really don't think it's going anywhere.
Reader was used and loved by a LOT of people WITHIN google, but it was shut down (and the leadership that loved it even made arguments in front of the company why it "had to be shut down").
AFAICT Scholar remains because Anurag built up massive cred in the early years (he was a critically important search engineer) with Larry Page and kept his infra costs and headcount really small, while also taking advantage of search infra).
If it matters, they cited declining usage of Reader as a reason for shutting it down.
It seems like Scholar has an overall upward trend, although their methodology notes make it hard to compare some periods directly:
https://trends.google.com/trends/explore?date=all&q=%2Fm%2F0...
I'm basically assuming this is the rate of growth of graduate school, and no competing products have had any real effect?
Reader usage was declining because the application was not being developed. The other reason they mentioned is that it would have required a lot of work to rewrite the app to be consistent with the new user data policies being put into place.
I'd not known about "F.D.C. Willard" — the nom de plume of a Michigan State physics professor's Siamese cat, Chester — who was listed as a co-author of a number of the professor's physics papers.
More on Chester and his co-author status: https://en.wikipedia.org/wiki/F._D._C._Willard
I did not know about PDF Scholar Readee extension [1]. Unfortunately the reason is that I use Firefox only (and safari iOS) and it is not available there. The AI outlines will be useful and I can think of myself using it.
I do not want to comment on number 20. I really wished that I joined CERN 10 years earlier but then it is the mistake of my parents :)
[1] https://chromewebstore.google.com/detail/google-scholar-pdf-...
Related: 2014 article by Steven Levy, titled "The Gentleman Who Made Scholar": https://www.wired.com/2014/10/the-gentleman-who-made-scholar...
> Would he want to continue working on Scholar for another ten years? “One always believes there are other opportunities, but the problem is how to pursue them when you are in a place you like and you have been doing really well. I can do problems that seem very interesting me — but the biggest impact I can possible make is helping people who are solving the world’s problems to be more efficient. If I can make the world’s researchers ten percent more efficient, consider the cumulative impact of that. So if I ended up spending the next ten years going this, I think I would be extremely happy.”
Has he still been working on it in the 10 years since this article? His name is in the byline of the new blog post, but it's not clear from that how much he's been working on it.
12-13 years ago, I ran the system that inlined Scholar and other results on the main search result pages. Anurag was still involved, but AFAIR Alex, the other author of the post who also had been there from the start, worked on most code changes. I would guess that things are more or less the same today. (Because it had such limited headcount, Scholar was known to lag behind other services when it came to code/infrastructure migrations.)
Thanks for that inside scoop, even if it's a bit dated; I wonder if they read this discussion, perhaps.
An important feature request would be a view where only peer-reviewed publications (specifically, not ArXiv and other pre-print archives) are included in the citation counts, and self-citations are also excluded.
A way to download all citation sources would also be a great nice-to-have.
Unpopular opinion but I really liked Microsoft Academic instead until they canned it, sadly.
What do you make of OpenAlex, which inherited the dataset?
I liked Microsoft Academic far better, if only because it actually had an API.
A reminder to everyone: if you want a "legal" copy of a paper you can always just try emailing one of the first authors. They will 99.99% send you back a PDF.
Our department uses GScholar as a great research-focused CV generator. Not used formally except that faculty pages have a link to their GS pages.
I wish GScholar wouldn't embrace bibliometrics so much. Sort papers by date (most recent papers first) by default on an author's page rather than by citation count, or at least give author the choice to individually opt-in to sort by date by default.
Google Scholar is fantastic stuff. I am so grateful for it. It’s crazy how easy it is to find papers these days by just going to it. University library search functions are completely useless in comparison.
> 1. The team started with just two of us.
My guess for a while has been that it was back to two of them! if that!
The most fun fact is that it still exists!
Some fun Google Scholar history from another perspective.
https://youtu.be/DZ2Bgwyx3nU?t=315
I recommend you watch the rest of the video, on the subject of open/closed and enclosure of infrastructure.
The post uses the expression "delve into" :-/
Is this a jokey reference to that time Paul Graham upset large amounts of Nigerians on Twitter? Or, rather, genuine concern at the thought that the article may have been generated by chatbots?
It´s because Taylor Swift´s lates album uses a lot of ´delve´.
Oh good, it's just a celebration and not an announcement that they're killing it.
I've been using Google Scholar for a long time, but I'm finding ChatGPT search with well-crafted prompts gets more focused and relevant results than a complex keyword search on GS does. However it's often still easier to find a link to the pdf version of the paper using GS, but then scihub is still an option and can work when all else fails.
Huh. I tried the "Listen to article" button, because I knew it was going to be generated and was curious to hear how it sounded.
Interestingly, it highlighted the words as it read. I haven't seen that before online. Not sure how useful it is (especially for anyone interested in this particular topic), but I thought it was a neat innovation nevertheless.
How long till they kill it?
21. No API
I was hoping it would be 20 tips and tricks on how to use the service better not random fun facts about its history :-(
For a second I thought this was buzzfeed for some reason.