|
Today I finished rewriting our search engine to use Sphinx RT indexes which means new content is indexed in real-time.
All new posts, edits, thread renames / moves, etc are now immediately visible in search results. While I was at it, I fixed a couple of the annoying limitations with the old search such as not being able to title search by username (eg, find all topics by a single user), and being unable to find many results when doing forum specific title searches.
There's also a preprocessing step that removes quotes and bbcode, so you should no longer find duplicate content results where people have quoted the same person 100 times.
Any problems / questions, please post!
|
sick more wizardly searches. thanks R1CH
|
ALLEYCAT BLUES50638 Posts
I approve of this change greatly.
|
Ah yeah, played around with testing sphinx performance and it's insanely quick. Nice changes, keep up the good work.
|
The new search works like magic! Awesome.
|
Thank you! I also approve of this change.
|
removing quotes is the best thing about this. good job
|
|
|
There's also a preprocessing step that removes quotes and bbcode, so you should no longer find duplicate content results where people have quoted the same person 100 times.
Does this mean you can hide a post from search by placing it in quotes?
Edit: Answer: Nope.
|
Rich making the site more awesome once again. Gj!
|
This is great! Thanks R1CH.
|
Much appreciated. Going to give it a spin now.
|
No more excuse for people creating the same thread again and again.
|
Netherlands19138 Posts
Thanks alot, I use the search function several times a day and pretty much everything you changed is a point of improvement for me.
|
|
|
FREEAGLELAND26782 Posts
|
United Kingdom14103 Posts
R1CH! <3
wait. R1CH does this mean I can't use the search function any more to check if someone has replied to a post of mine?
|
|
|
very nice R1CH. Thanks as always.
|
hm now i cant search my username to see whos quoted & responded to me...
|
Thanks for improving the site as always, R1CH! The search seems really efficient now.
I will say that I'll miss being able to search quoted text, however. In fast moving LR threads I often miss replies to my posts, so I'd just do a content search for my name to find any posts that had quoted mine. I imagine I'm in a small minority of posters who used the search in this way, though.
|
United Kingdom14103 Posts
On September 21 2013 02:10 GulpyBlinkeyes wrote:
Thanks for improving the site as always, R1CH! The search seems really efficient now.
I will say that I'll miss being able to search quoted text, however. In fast moving LR threads I often miss replies to my posts, so I'd just do a content search for my name to find any posts that had quoted mine. I imagine I'm in a small minority of posters who used the search in this way, though.
I used this too :/
|
R1CH, you're a wizard and a scholar. Thanks!
|
|
|
|
|
Hong Kong9169 Posts
|
Excellent, some of the search limitations have always bothered me in the past, thanks a lot!
|
On September 21 2013 02:10 GulpyBlinkeyes wrote:
Thanks for improving the site as always, R1CH! The search seems really efficient now.
I will say that I'll miss being able to search quoted text, however. In fast moving LR threads I often miss replies to my posts, so I'd just do a content search for my name to find any posts that had quoted mine. I imagine I'm in a small minority of posters who used the search in this way, though.
Get TL+ and then use the "all" function and use your browser's search function.
It's a clever ploy to encourage people to get TL+
|
Thanks R1CH! Easier and better navigation is always a plus! :D
|
I think the being in real time is a huge improvement. this is nice.
|
when are you going to update the forum ui and functionality so it's not like it's from '98?
User was temp banned for this post.
|
|
|
Great work R1CH.
Maybe a separate part of the site for people that have quoted a user would be a good idea?
|
R1CH -- how big are the servers running TL? Sphinx likes to eat ram!
|
|
|
Very nice indeed! Thanks a lot! 
On September 21 2013 05:29 yesrr wrote:
when are you going to update the forum ui and functionality so it's not like it's from '98?
User was temp banned for this post. lol :')
|
I love you so much R1CH. Thank you.
|
Yeah, I definitely notice an improvement in search
|
|
|
On September 21 2013 02:08 synapse wrote:
hm now i cant search my username to see whos quoted & responded to me...
Yeah same. Like that its real time though
On September 21 2013 04:36 Lonyo wrote:Show nested quote +On September 21 2013 02:10 GulpyBlinkeyes wrote:
Thanks for improving the site as always, R1CH! The search seems really efficient now.
I will say that I'll miss being able to search quoted text, however. In fast moving LR threads I often miss replies to my posts, so I'd just do a content search for my name to find any posts that had quoted mine. I imagine I'm in a small minority of posters who used the search in this way, though. Get TL+ and then use the "all" function and use your browser's search function. It's a clever ploy to encourage people to get TL+
Disagree - Even if I had TL+ I don't want to have to go back to each thread I post in to see if someone quoted me. I'd rather just have all that information garnered in one location like it was before.
|
On September 21 2013 09:23 Kal_rA wrote:Show nested quote +On September 21 2013 02:08 synapse wrote:
hm now i cant search my username to see whos quoted & responded to me... Yeah same. Like that its real time though Show nested quote +On September 21 2013 04:36 Lonyo wrote:On September 21 2013 02:10 GulpyBlinkeyes wrote:
Thanks for improving the site as always, R1CH! The search seems really efficient now.
I will say that I'll miss being able to search quoted text, however. In fast moving LR threads I often miss replies to my posts, so I'd just do a content search for my name to find any posts that had quoted mine. I imagine I'm in a small minority of posters who used the search in this way, though. Get TL+ and then use the "all" function and use your browser's search function. It's a clever ploy to encourage people to get TL+ Disagree - Even if I had TL+ I don't want to have to go back to each thread I post in to see if someone quoted me. I'd rather just have all that information garnered in one location like it was before.
I have TL+ and I don't particularly like that option :[ I guess it'll have to do, but just searching your name for replies was way easier. I used that a lot.
|
Canada11439 Posts
eg, find all topics by a single user), and being unable to find many results when doing forum specific title searches.
It's the best! I love it. There are SO many times where I can remember the user that created the thread, but not the thread title.
|
On September 21 2013 07:20 Yorbon wrote:Very nice indeed! Thanks a lot! Show nested quote +On September 21 2013 05:29 yesrr wrote:
when are you going to update the forum ui and functionality so it's not like it's from '98?
User was temp banned for this post. lol :')
I can now search to find the post that got him banned, and not just everybody quoting the [b][red]User was ...
Brilliant!
|
On September 21 2013 02:08 synapse wrote:
hm now i cant search my username to see whos quoted & responded to me...
Muhahahahahah you'll never know about this !
Tought it was an interesting point you bring, the only possible downside up to now. Not much harm compared to the advantages, but does not refrain from looking for a solution.
------
Oh dam, now it's gonna be hard to start a topic with '' Excuse me I did not find a... '''
|
|
|
United Kingdom10823 Posts
I didn't understand most oft hose words, but I'm oddly turned on
|
On September 21 2013 13:11 Tachion wrote:Show nested quote +On September 21 2013 09:23 Kal_rA wrote:On September 21 2013 02:08 synapse wrote:
hm now i cant search my username to see whos quoted & responded to me... Yeah same. Like that its real time though On September 21 2013 04:36 Lonyo wrote:On September 21 2013 02:10 GulpyBlinkeyes wrote:
Thanks for improving the site as always, R1CH! The search seems really efficient now.
I will say that I'll miss being able to search quoted text, however. In fast moving LR threads I often miss replies to my posts, so I'd just do a content search for my name to find any posts that had quoted mine. I imagine I'm in a small minority of posters who used the search in this way, though. Get TL+ and then use the "all" function and use your browser's search function. It's a clever ploy to encourage people to get TL+ Disagree - Even if I had TL+ I don't want to have to go back to each thread I post in to see if someone quoted me. I'd rather just have all that information garnered in one location like it was before. I have TL+ and I don't particularly like that option :[ I guess it'll have to do, but just searching your name for replies was way easier. I used that a lot.
Exactly. There is no way I could have replied to this right here if I didn't read through the thread again manually. The way it was before was working really well imo. Browsing TL just got harder 
|
On September 21 2013 01:05 Ghanburighan wrote:Show nested quote +There's also a preprocessing step that removes quotes and bbcode, so you should no longer find duplicate content results where people have quoted the same person 100 times. Does this mean you can hide a post from search by placing it in quotes? Edit: Answer: Nope.
What do you think this is, amateur hour? R1CH got dis shit yo.
Thanks R1CHie.
|
|
|
Awesome, good work R1CH
|
On September 22 2013 03:30 Kal_rA wrote:Show nested quote +On September 21 2013 13:11 Tachion wrote:On September 21 2013 09:23 Kal_rA wrote:On September 21 2013 02:08 synapse wrote:
hm now i cant search my username to see whos quoted & responded to me... Yeah same. Like that its real time though On September 21 2013 04:36 Lonyo wrote:On September 21 2013 02:10 GulpyBlinkeyes wrote:
Thanks for improving the site as always, R1CH! The search seems really efficient now.
I will say that I'll miss being able to search quoted text, however. In fast moving LR threads I often miss replies to my posts, so I'd just do a content search for my name to find any posts that had quoted mine. I imagine I'm in a small minority of posters who used the search in this way, though. Get TL+ and then use the "all" function and use your browser's search function. It's a clever ploy to encourage people to get TL+ Disagree - Even if I had TL+ I don't want to have to go back to each thread I post in to see if someone quoted me. I'd rather just have all that information garnered in one location like it was before. I have TL+ and I don't particularly like that option :[ I guess it'll have to do, but just searching your name for replies was way easier. I used that a lot. Exactly. There is no way I could have replied to this right here if I didn't read through the thread again manually. The way it was before was working really well imo. Browsing TL just got harder
so is there no other way to do the same thing? i just wanna see who responds to me T_T
|
When are you getting elected for president?
|
Small update: Square brackets are now excluded from title searches, eg [GOM] and GOM will produce identical results.
|
Bearded Elder29904 Posts
R1CH You're such a boss. Appreciated.
|
hrm, not being able to find people who have quoted you makes things a bit more complicated. still, well done.
|
Did this site use SQL Full Text Search prior to this, or SQL LIKE's if you will? It would be quite a feat to have used it for this long with such a website, while not lagging up the DB constantly. When I had to deploy Sphinx SE a couple of times 7 years ago it was for websites with way less docs, and they were in dire need.
|
We used to use MySQL fulltext up until 2010 when I replaced it with incremental Sphinx. It used to kill the server for 30 seconds any time someone did a search .
|
|
|
Update: the same technology has now been applied to private message searching! You can now find messages by contents instead of just the title.
|
Small update: some popular threads were causing filtering issues (eg the random pics thread), these have been fixed so search results should include all matching topics now.
|
|
|
|
|
|
EPT
