Non-English Characters

Hello all,
We are developing an add-on for GoldMine, however my problem is a
generic SQL Server problem. The situation is this; we have a database,
with collation set to SQL Latin. When we connect to database via
Goldmine, Enterprise Manager or Delphi (through ADO) we cannot see
Turkish characters. When I set SQL Server machine's "Language For
Non-Unicode Programs" setting to Turkish, GoldMine works fine, however
there is still problem in EM and Delphi.
Does anybody know how to solve this issue?(saygin@.gmail.com) writes:
> We are developing an add-on for GoldMine, however my problem is a
> generic SQL Server problem. The situation is this; we have a database,
> with collation set to SQL Latin. When we connect to database via
> Goldmine, Enterprise Manager or Delphi (through ADO) we cannot see
> Turkish characters. When I set SQL Server machine's "Language For
> Non-Unicode Programs" setting to Turkish, GoldMine works fine, however
> there is still problem in EM and Delphi.
> Does anybody know how to solve this issue?

What datatype does the columns have? varchar or nvarchar?

Exactly which collation are you using? There are a whole number of
collations with the name starting with "SQL_Latin".

Do all Turkish characters get mashed? Or only some? Does (lowercase C with
cedilla) work? If not, how do you see it in EM? How does the dotless
lowercase i get represented?

Which client API do you use in Delphi?

--
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pr...oads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodin...ions/books.mspx|||Hi,
Exact collation is SQL_Latin1_General_CP1_CI_AS.
Not all characters are mashed, (c with credila) is ok, however
dotless i becomes y. (u with dots) are ok, however s (s with
credila) and g (g with ~ over it) becomes ? (question mark)|||And by the way, I use ADO to connect with Delphi.|||iSt (saygin@.gmail.com) writes:
> Exact collation is SQL_Latin1_General_CP1_CI_AS.
> Not all characters are mashed, (c with credila) is ok, however
> dotless i becomes y. (u with dots) are ok, however s (s with
> credila) and g (g with ~ over it) becomes ? (question mark)

Are you using varchar or nvarchar?

If you are using varchar you are fighting an uphill battle. The code
page SQL_Latin1_General_CP1_CI_AS is CP1252 which is not the best
code page for Turkish.

You should use a collation with code page 1254, for instance
SQL_Latin1_General_CP1254_CI_AS or any of the collations with
the name starting with Turkish.

--
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pr...oads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodin...ions/books.mspx|||Hi,
DB comes default with this collation. And if I offer to change, I can
be asked how GoldMine can show TR characters without problem.|||Columns are varchar by the way...|||iSt (saygin@.gmail.com) writes:
> DB comes default with this collation. And if I offer to change, I can
> be asked how GoldMine can show TR characters without problem.

I don't know what means of connection GoldMine uses. I have not heard of
the program before.

But when you connect through most APIs, they will perform a character
conversion to similar characters. The one API where this does not happen
is the old DB-Library.

By the way, if you got that collation installed by default, this means
that your operating system has US English as its default language. Had
the OS had Turkish as its default language, you would had been offered
Turkish_CI_AS as your default collation.
--
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pr...oads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodin...ions/books.mspx

Non English Characters

Hi,
We are developing a small web interface to a local ERP software, which
uses SQL Server 2000 as database. The database uses SQL_Latin1_CP1
collation, and the fields are varchar (not nvarchar), however, the main
program inserts and reads non-English (Turkish) characters into these
columns. However, when we connect to database with ADO.NET, these
characters are not read correctly. (The situation is same when I check
tables with Enterprise Manager and Query Analyzer)

In a past situation (which was about a Win32 application), I have heard
about character conversion behaviour of ADO (and many other DB
libraries) and solved that problem using BDE instead of ADO, so that
the connection is made via DB-Library instead of OLEDB.

But this way cannot be applied to my ASP.NET situation, and there is NO
way to change database collation. Must I use a ADO.NET property, or use
another provider, or maybe another library? Any advices? Thanks...Set your culture settings in your web.config file for the appropriate code
set that you use. This should solve the problem. You can also cast the
characters to ASCII by doing this (courtesy or Erland Sommerskog)

SELECT convert(varchar(6), N'IGDEM' COLLATE Cyrillic_General_CS_AS)

--
Hilary Cotter
Director of Text Mining and Database Strategy
RelevantNOISE.Com - Dedicated to mining blogs for business intelligence.

This posting is my own and doesn't necessarily represent RelevantNoise's
positions, strategies or opinions.

Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
<saygin@.gmail.comwrote in message
news:1159361639.289293.15470@.d34g2000cwd.googlegro ups.com...

Quote:

Originally Posted by

Hi,
We are developing a small web interface to a local ERP software, which
uses SQL Server 2000 as database. The database uses SQL_Latin1_CP1
collation, and the fields are varchar (not nvarchar), however, the main
program inserts and reads non-English (Turkish) characters into these
columns. However, when we connect to database with ADO.NET, these
characters are not read correctly. (The situation is same when I check
tables with Enterprise Manager and Query Analyzer)
>
In a past situation (which was about a Win32 application), I have heard
about character conversion behaviour of ADO (and many other DB
libraries) and solved that problem using BDE instead of ADO, so that
the connection is made via DB-Library instead of OLEDB.
>
But this way cannot be applied to my ASP.NET situation, and there is NO
way to change database collation. Must I use a ADO.NET property, or use
another provider, or maybe another library? Any advices? Thanks...
>

|||A couple more thoughts are you storing the strings in nVarchar(20) data type
columns. Also if you use Erland's method you will loose the Turkish
characters if you store them using his technique but the searching should
work.

--
Hilary Cotter
Director of Text Mining and Database Strategy
RelevantNOISE.Com - Dedicated to mining blogs for business intelligence.

This posting is my own and doesn't necessarily represent RelevantNoise's
positions, strategies or opinions.

Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
<saygin@.gmail.comwrote in message
news:1159361639.289293.15470@.d34g2000cwd.googlegro ups.com...

Quote:

Originally Posted by

Hi,
We are developing a small web interface to a local ERP software, which
uses SQL Server 2000 as database. The database uses SQL_Latin1_CP1
collation, and the fields are varchar (not nvarchar), however, the main
program inserts and reads non-English (Turkish) characters into these
columns. However, when we connect to database with ADO.NET, these
characters are not read correctly. (The situation is same when I check
tables with Enterprise Manager and Query Analyzer)
>
In a past situation (which was about a Win32 application), I have heard
about character conversion behaviour of ADO (and many other DB
libraries) and solved that problem using BDE instead of ADO, so that
the connection is made via DB-Library instead of OLEDB.
>
But this way cannot be applied to my ASP.NET situation, and there is NO
way to change database collation. Must I use a ADO.NET property, or use
another provider, or maybe another library? Any advices? Thanks...
>

|||Hi, thanks for your interest.
Changing culture setting did not help, also there is no chance to
change data type of columns, they are varchar, not nvarchar.
Character conversion of database libraries and using a DB-Library based
API (in my previous example I'd chosen BDE) was Mr.Erland's
recommendation also :) However I cannot find a similar solution for
ASP.NET

Hilary Cotter wrote:

Quote:

Originally Posted by

A couple more thoughts are you storing the strings in nVarchar(20) data type
columns. Also if you use Erland's method you will loose the Turkish
characters if you store them using his technique but the searching should
work.
>
--
Hilary Cotter
Director of Text Mining and Database Strategy
RelevantNOISE.Com - Dedicated to mining blogs for business intelligence.
>
This posting is my own and doesn't necessarily represent RelevantNoise's
positions, strategies or opinions.
>
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
>
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
>
>
>
<saygin@.gmail.comwrote in message
news:1159361639.289293.15470@.d34g2000cwd.googlegro ups.com...

Quote:

Originally Posted by

Hi,
We are developing a small web interface to a local ERP software, which
uses SQL Server 2000 as database. The database uses SQL_Latin1_CP1
collation, and the fields are varchar (not nvarchar), however, the main
program inserts and reads non-English (Turkish) characters into these
columns. However, when we connect to database with ADO.NET, these
characters are not read correctly. (The situation is same when I check
tables with Enterprise Manager and Query Analyzer)

In a past situation (which was about a Win32 application), I have heard
about character conversion behaviour of ADO (and many other DB
libraries) and solved that problem using BDE instead of ADO, so that
the connection is made via DB-Library instead of OLEDB.

But this way cannot be applied to my ASP.NET situation, and there is NO
way to change database collation. Must I use a ADO.NET property, or use
another provider, or maybe another library? Any advices? Thanks...

|||Hi,
Choosing ODBC provider rather than OLEDB, and turning of character
translation option solved my problem.

saygin@.gmail.com wrote:

Quote:

Originally Posted by

Hi, thanks for your interest.
Changing culture setting did not help, also there is no chance to
change data type of columns, they are varchar, not nvarchar.
Character conversion of database libraries and using a DB-Library based
API (in my previous example I'd chosen BDE) was Mr.Erland's
recommendation also :) However I cannot find a similar solution for
ASP.NET
>
>
>
>
Hilary Cotter wrote:

Quote:

Originally Posted by

A couple more thoughts are you storing the strings in nVarchar(20) data type
columns. Also if you use Erland's method you will loose the Turkish
characters if you store them using his technique but the searching should
work.

--
Hilary Cotter
Director of Text Mining and Database Strategy
RelevantNOISE.Com - Dedicated to mining blogs for business intelligence.

This posting is my own and doesn't necessarily represent RelevantNoise's
positions, strategies or opinions.

Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
<saygin@.gmail.comwrote in message
news:1159361639.289293.15470@.d34g2000cwd.googlegro ups.com...

Quote:

Originally Posted by

Hi,
We are developing a small web interface to a local ERP software, which
uses SQL Server 2000 as database. The database uses SQL_Latin1_CP1
collation, and the fields are varchar (not nvarchar), however, the main
program inserts and reads non-English (Turkish) characters into these
columns. However, when we connect to database with ADO.NET, these
characters are not read correctly. (The situation is same when I check
tables with Enterprise Manager and Query Analyzer)
>
In a past situation (which was about a Win32 application), I have heard
about character conversion behaviour of ADO (and many other DB
libraries) and solved that problem using BDE instead of ADO, so that
the connection is made via DB-Library instead of OLEDB.
>
But this way cannot be applied to my ASP.NET situation, and there is NO
way to change database collation. Must I use a ADO.NET property, or use
another provider, or maybe another library? Any advices? Thanks...
>

Non English Characters

Hi,
We are developing a small web interface to a local ERP software, which
uses SQL Server 2000 as database. The database uses SQL_Latin1_CP1
collation, and the fields are varchar (not nvarchar), however, the main
program inserts and reads non-English (Turkish) characters into these
columns. However, when we connect to database with ADO.NET, these
characters are not read correctly. (The situation is same when I check
tables with Enterprise Manager and Query Analyzer)
In a past situation (which was about a Win32 application), I have heard
about character conversion behaviour of ADO (and many other DB
libraries) and solved that problem using BDE instead of ADO, so that
the connection is made via DB-Library instead of OLEDB.
But this way cannot be applied to my ASP.NET situation, and there is NO
way to change database collation. Must I use a ADO.NET property, or use
another provider, or maybe another library? Any advices? Thanks...
Set your culture settings in your web.config file for the appropriate code
set that you use. This should solve the problem. You can also cast the
characters to ASCII by doing this (courtesy or Erland Sommerskog)
SELECT convert(varchar(6), N'IGDEM' COLLATE Cyrillic_General_CS_AS)
Hilary Cotter
Director of Text Mining and Database Strategy
RelevantNOISE.Com - Dedicated to mining blogs for business intelligence.
This posting is my own and doesn't necessarily represent RelevantNoise's
positions, strategies or opinions.
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
<saygin@.gmail.com> wrote in message
news:1159361639.289293.15470@.d34g2000cwd.googlegro ups.com...
> Hi,
> We are developing a small web interface to a local ERP software, which
> uses SQL Server 2000 as database. The database uses SQL_Latin1_CP1
> collation, and the fields are varchar (not nvarchar), however, the main
> program inserts and reads non-English (Turkish) characters into these
> columns. However, when we connect to database with ADO.NET, these
> characters are not read correctly. (The situation is same when I check
> tables with Enterprise Manager and Query Analyzer)
> In a past situation (which was about a Win32 application), I have heard
> about character conversion behaviour of ADO (and many other DB
> libraries) and solved that problem using BDE instead of ADO, so that
> the connection is made via DB-Library instead of OLEDB.
> But this way cannot be applied to my ASP.NET situation, and there is NO
> way to change database collation. Must I use a ADO.NET property, or use
> another provider, or maybe another library? Any advices? Thanks...
>
|||A couple more thoughts are you storing the strings in nVarchar(20) data type
columns. Also if you use Erland's method you will loose the Turkish
characters if you store them using his technique but the searching should
work.
Hilary Cotter
Director of Text Mining and Database Strategy
RelevantNOISE.Com - Dedicated to mining blogs for business intelligence.
This posting is my own and doesn't necessarily represent RelevantNoise's
positions, strategies or opinions.
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
<saygin@.gmail.com> wrote in message
news:1159361639.289293.15470@.d34g2000cwd.googlegro ups.com...
> Hi,
> We are developing a small web interface to a local ERP software, which
> uses SQL Server 2000 as database. The database uses SQL_Latin1_CP1
> collation, and the fields are varchar (not nvarchar), however, the main
> program inserts and reads non-English (Turkish) characters into these
> columns. However, when we connect to database with ADO.NET, these
> characters are not read correctly. (The situation is same when I check
> tables with Enterprise Manager and Query Analyzer)
> In a past situation (which was about a Win32 application), I have heard
> about character conversion behaviour of ADO (and many other DB
> libraries) and solved that problem using BDE instead of ADO, so that
> the connection is made via DB-Library instead of OLEDB.
> But this way cannot be applied to my ASP.NET situation, and there is NO
> way to change database collation. Must I use a ADO.NET property, or use
> another provider, or maybe another library? Any advices? Thanks...
>
|||Hi, thanks for your interest.
Changing culture setting did not help, also there is no chance to
change data type of columns, they are varchar, not nvarchar.
Character conversion of database libraries and using a DB-Library based
API (in my previous example I'd chosen BDE) was Mr.Erland's
recommendation also However I cannot find a similar solution for
ASP.NET
Hilary Cotter wrote:[vbcol=seagreen]
> A couple more thoughts are you storing the strings in nVarchar(20) data type
> columns. Also if you use Erland's method you will loose the Turkish
> characters if you store them using his technique but the searching should
> work.
> --
> Hilary Cotter
> Director of Text Mining and Database Strategy
> RelevantNOISE.Com - Dedicated to mining blogs for business intelligence.
> This posting is my own and doesn't necessarily represent RelevantNoise's
> positions, strategies or opinions.
> Looking for a SQL Server replication book?
> http://www.nwsu.com/0974973602.html
> Looking for a FAQ on Indexing Services/SQL FTS
> http://www.indexserverfaq.com
>
> <saygin@.gmail.com> wrote in message
> news:1159361639.289293.15470@.d34g2000cwd.googlegro ups.com...
|||Hi,
Choosing ODBC provider rather than OLEDB, and turning of character
translation option solved my problem.
saygin@.gmail.com wrote:[vbcol=seagreen]
> Hi, thanks for your interest.
> Changing culture setting did not help, also there is no chance to
> change data type of columns, they are varchar, not nvarchar.
> Character conversion of database libraries and using a DB-Library based
> API (in my previous example I'd chosen BDE) was Mr.Erland's
> recommendation also However I cannot find a similar solution for
> ASP.NET
>
>
> Hilary Cotter wrote:

Non English Characters

Hi,
We are developing a small web interface to a local ERP software, which
uses SQL Server 2000 as database. The database uses SQL_Latin1_CP1
collation, and the fields are varchar (not nvarchar), however, the main
program inserts and reads non-English (Turkish) characters into these
columns. However, when we connect to database with ADO.NET, these
characters are not read correctly. (The situation is same when I check
tables with Enterprise Manager and Query Analyzer)
In a past situation (which was about a Win32 application), I have heard
about character conversion behaviour of ADO (and many other DB
libraries) and solved that problem using BDE instead of ADO, so that
the connection is made via DB-Library instead of OLEDB.
But this way cannot be applied to my ASP.NET situation, and there is NO
way to change database collation. Must I use a ADO.NET property, or use
another provider, or maybe another library? Any advices? Thanks...Set your culture settings in your web.config file for the appropriate code
set that you use. This should solve the problem. You can also cast the
characters to ASCII by doing this (courtesy or Erland Sommerskog)
SELECT convert(varchar(6), N'IGDEM' COLLATE Cyrillic_General_CS_AS)
Hilary Cotter
Director of Text Mining and Database Strategy
RelevantNOISE.Com - Dedicated to mining blogs for business intelligence.
This posting is my own and doesn't necessarily represent RelevantNoise's
positions, strategies or opinions.
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
<saygin@.gmail.com> wrote in message
news:1159361639.289293.15470@.d34g2000cwd.googlegroups.com...
> Hi,
> We are developing a small web interface to a local ERP software, which
> uses SQL Server 2000 as database. The database uses SQL_Latin1_CP1
> collation, and the fields are varchar (not nvarchar), however, the main
> program inserts and reads non-English (Turkish) characters into these
> columns. However, when we connect to database with ADO.NET, these
> characters are not read correctly. (The situation is same when I check
> tables with Enterprise Manager and Query Analyzer)
> In a past situation (which was about a Win32 application), I have heard
> about character conversion behaviour of ADO (and many other DB
> libraries) and solved that problem using BDE instead of ADO, so that
> the connection is made via DB-Library instead of OLEDB.
> But this way cannot be applied to my ASP.NET situation, and there is NO
> way to change database collation. Must I use a ADO.NET property, or use
> another provider, or maybe another library? Any advices? Thanks...
>|||A couple more thoughts are you storing the strings in nVarchar(20) data type
columns. Also if you use Erland's method you will loose the Turkish
characters if you store them using his technique but the searching should
work.
Hilary Cotter
Director of Text Mining and Database Strategy
RelevantNOISE.Com - Dedicated to mining blogs for business intelligence.
This posting is my own and doesn't necessarily represent RelevantNoise's
positions, strategies or opinions.
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
<saygin@.gmail.com> wrote in message
news:1159361639.289293.15470@.d34g2000cwd.googlegroups.com...
> Hi,
> We are developing a small web interface to a local ERP software, which
> uses SQL Server 2000 as database. The database uses SQL_Latin1_CP1
> collation, and the fields are varchar (not nvarchar), however, the main
> program inserts and reads non-English (Turkish) characters into these
> columns. However, when we connect to database with ADO.NET, these
> characters are not read correctly. (The situation is same when I check
> tables with Enterprise Manager and Query Analyzer)
> In a past situation (which was about a Win32 application), I have heard
> about character conversion behaviour of ADO (and many other DB
> libraries) and solved that problem using BDE instead of ADO, so that
> the connection is made via DB-Library instead of OLEDB.
> But this way cannot be applied to my ASP.NET situation, and there is NO
> way to change database collation. Must I use a ADO.NET property, or use
> another provider, or maybe another library? Any advices? Thanks...
>|||Hi, thanks for your interest.
Changing culture setting did not help, also there is no chance to
change data type of columns, they are varchar, not nvarchar.
Character conversion of database libraries and using a DB-Library based
API (in my previous example I'd chosen BDE) was Mr.Erland's
recommendation also However I cannot find a similar solution for
ASP.NET
Hilary Cotter wrote:[vbcol=seagreen]
> A couple more thoughts are you storing the strings in nVarchar(20) data ty
pe
> columns. Also if you use Erland's method you will loose the Turkish
> characters if you store them using his technique but the searching should
> work.
> --
> Hilary Cotter
> Director of Text Mining and Database Strategy
> RelevantNOISE.Com - Dedicated to mining blogs for business intelligence.
> This posting is my own and doesn't necessarily represent RelevantNoise's
> positions, strategies or opinions.
> Looking for a SQL Server replication book?
> http://www.nwsu.com/0974973602.html
> Looking for a FAQ on Indexing Services/SQL FTS
> http://www.indexserverfaq.com
>
> <saygin@.gmail.com> wrote in message
> news:1159361639.289293.15470@.d34g2000cwd.googlegroups.com...|||Hi,
Choosing ODBC provider rather than OLEDB, and turning of character
translation option solved my problem.
saygin@.gmail.com wrote:[vbcol=seagreen]
> Hi, thanks for your interest.
> Changing culture setting did not help, also there is no chance to
> change data type of columns, they are varchar, not nvarchar.
> Character conversion of database libraries and using a DB-Library based
> API (in my previous example I'd chosen BDE) was Mr.Erland's
> recommendation also However I cannot find a similar solution for
> ASP.NET
>
>
> Hilary Cotter wrote: