wondering if there is any plugin for searching chinese

tianyi · Post by **tianyi** » Fri Apr 10, 2009 12:18 pm

I am a user from China, and I need a plugin for searching chinese in my blog. Can anyone develop a plugin like this?

judebert · Post by **judebert** » Fri Apr 10, 2009 5:55 pm

It was my impression that the Quicksearch plugin would search your blog regardless of language, as long as your database and blog use the same character encoding. Are you actually having problems with searching?

tianyi · Post by **tianyi** » Mon Apr 13, 2009 4:22 am

judebert wrote:It was my impression that the Quicksearch plugin would search your blog regardless of language, as long as your database and blog use the same character encoding. Are you actually having problems with searching?

Yes, I do have the difficulties in searching. xu-kaidotcom is my own blog, at the right column of the blog is the quick search plugin. However, can not search any chinese character. It is an problem since long ago. I guess that nobody came here to report this problem, but modify the codes.

/include/functions_entries.inc.php

Code: Select all

if (preg_match('@["+-*~<>()]+@', $term)) {
$cond['find_part'] = "MATCH(title,body,extended) AGAINST('$term' IN BOOLEAN MODE)";
} else {
$cond['find_part'] = "MATCH(title,body,extended) AGAINST('$term')";
}

change to

Code: Select all

if(preg_match("/^[x80-xff]+$/", $term))
{
$cond['find_part'] = "((e.title LIKE ('%" . addslashes($term) . "%')) or (e.body LIKE ('%" . addslashes($term) . "%')) or (e.extended LIKE ('%" . addslashes($term) . "%')))";
}
else
{
if (preg_match('@["+-*~<>()]+@', $term)) {
$cond['find_part'] = "MATCH(title,body,extended) AGAINST('$term' IN BOOLEAN MODE)";
} else {
$cond['find_part'] = "MATCH(title,body,extended) AGAINST('$term')";
}
}

tianyi · Post by **tianyi** » Mon Apr 13, 2009 4:30 am

This method is available but not a good way.
Modifying the code is not convinient and the result is not exactly what i want.
I hope that there is a plugin to solve this problem

Post by **garvinhicking** » Mon Apr 13, 2009 1:42 pm

Hi!

Are you using the most recent MySQL version? This one should have charset support, and if your database is in UTF-8 format, your chinese characters should be included there and be no different than german umlauts?!

Regards,
Garvin

tianyi · Post by **tianyi** » Tue Apr 14, 2009 2:48 am

garvinhicking wrote:Hi!

Are you using the most recent MySQL version? This one should have charset support, and if your database is in UTF-8 format, your chinese characters should be included there and be no different than german umlauts?!

Regards,
Garvin

Mysql Version: 5.0.51a-community
I think it is a new one, thought not a latest one.

tianyi · Post by **tianyi** » Wed Apr 15, 2009 7:19 am

Hello, anyone to answer my question?

Post by **garvinhicking** » Wed Apr 15, 2009 10:00 am

Hi!

tianyi wrote:Hello, anyone to answer my question?

Sadly not; only that quicksearch should work with MATCH AGAINST. Maybe mysql specific forums could help here.

Sadly I do not know any chinese or japanese, so I have no way of testing this...

Regards,
Garvin

judebert · Post by **judebert** » Wed Apr 15, 2009 6:17 pm

Perhaps the obvious question: Are your blog and your database both using UTF-8?

tianyi · Post by **tianyi** » Fri Apr 17, 2009 2:28 am

judebert wrote:Perhaps the obvious question: Are your blog and your database both using UTF-8?

: mysqlutf8.png (1.59 KiB) Viewed 7031 times

ayamico · Post by **ayamico** » Wed Feb 03, 2010 2:18 pm

I have been using serendipity for 5 years. Chinese searching has never worked. I decided to do some searching on why it does not work.

It looks like it is because mySQL full text search mode cannot support Chinese / Japanese character searching.

What I have found is "For FULLTEXT searches, we need to know where words begin and end. With Western languages, this is rarely a problem because most (if not all) of these use an easy-to-identify word boundary — the space character. However, this is not usually the case with Asian writing. We could use arbitrary halfway measures, like assuming that all Han characters represent words, or (for Japanese) depending on changes from Katakana to Hiragana due to grammatical endings. However, the only sure solution requires a comprehensive word list, which means that we would have to include a dictionary in the server for each Asian language supported."

http://blogs.sun.com/soapbox/entry/full ... uages_with

I can understand now why the following code modification is made

Code: Select all

if(preg_match("/^[x80-xff]+$/", $term))
{
$cond['find_part'] = "((e.title LIKE ('%" . addslashes($term) . "%')) or (e.body LIKE ('%" . addslashes($term) . "%')) or (e.extended LIKE ('%" . addslashes($term) . "%')))";
}
else
{
if (preg_match('@["+-*~<>()]+@', $term)) {
$cond['find_part'] = "MATCH(title,body,extended) AGAINST('$term' IN BOOLEAN MODE)";
} else {
$cond['find_part'] = "MATCH(title,body,extended) AGAINST('$term')";
}
}

Looking at the current serendipity codes, the above changes cannot be made in a plugin.

Any suggestion ?

wondering if there is any plugin for searching chinese

wondering if there is any plugin for searching chinese

Re: wondering if there is any plugin for searching chinese

Re: wondering if there is any plugin for searching chinese

Re: wondering if there is any plugin for searching chinese

Re: wondering if there is any plugin for searching chinese

Re: wondering if there is any plugin for searching chinese

Re: wondering if there is any plugin for searching chinese

Re: wondering if there is any plugin for searching chinese

Re: wondering if there is any plugin for searching chinese

Re: wondering if there is any plugin for searching chinese

Re: wondering if there is any plugin for searching chinese