Introducing chinese patents into PATSTAT

PATSTAT documentation says the raw data cover more than 90 patent office.
The question is how, and at what extent. In this post I start an investigation about SIPO patents that will last for some weeks.

The EPO’s co-operation with the State Intellectual Property Office of the People’s Republic of China (SIPO) is based on an agreement signed on 11 June 1985. An EPO-SIPO Joint Committee meets once a year to review relations and approve the following year’s program of activities. The co-operation was significantly enhanced in 1997 when EPOQUE ( an international search database for patents)  was made available to SIPO.
The overall objective is to facilitate trade between China and Europe by supporting China’s efforts to develop a modern and effective system for IPR protection. EPO aims to improve the quality of SIPO work as the Chinese patent system expands. 

To ensure equal quality standards of granted patents , the majority of the SIPO's 2,000+ patent examiners have been trained by the EPO. An additional 60 examiners per year receive training at EPO centres in The Hague, Munich and Vienna.

Since mid-1990s, the SIPO adopted the EPOQUE system,and as a result compatibility of archived patent information is now the basis of all digital data exchange.

EPO and the SIPO share patent databases with synchronization at weekly intervals. The SIPO has also recently opened its 32 000-entry database on traditional Chinese medicine to the EPO and its member states.

Here is a selection of the most relevant INPADOC-Newsletters referring to Chinese patent data, kind codes and number formats used in the INPADOC database (the links below are pointing to PDF files):

Newsflash 2005/16: Completion of Load of Backlogged Chinese Data (PDF)
Inpadoc Patent Gazette Newsletter 01/05 (PDF)
Inpadoc Patent Gazette Newsletter 02/04 (PDF)
Newsflash 2005/05: New CN Application number format and updates of Chinese bibliographic data (PDF)
Newsflash 2005/02: Change in kind codes of Chinese documents (PDF)
IPG-Newsletter 99/10: Translation of PCT application into Chinese language (PDF)
Newsflash 2005/25: Chinese application numbers (continued) (PDF)
Newsflash 2006/14: Reload of Chinese A, C and Y documents intended for the inclusion of Chinese language elements (PDF)
Newsflash 2004/17: Legal Status Data from Hong Kong (PDF)
Patent Information Supplement 09/2009
Newsflash 11/2009: Chinese legal status data in the EPO INPADOC database
Newsflash 11a/2009: Chinese legal status data in the EPO INPADOC database (update)
IPG-Newsletter 3/2005

Also EPO FAQ about china patents may be useful

  In mid-2009, the EPO has started uploading English legal status data for both Chinese patents and utility models documents as part of the INPADOC legal status information in esp@cenet.
The data upload commenced in week 18/2009;  the collection should include over 6.7 million records and cover all Chinese legal status events for patent and utility model applications in China dating back to October 1985.
 
You can obtain legal status information in English via the Chinese Patent Office's (SIPO) databases on the China Intellectual Property Net . All the above-mentioned sources there is a time delay of several months for legal status information in English.

data from SIPO: Annual Applications for Patents 

Year	Total
1999	134239
2000	170682
2001	203573
2002	252631
2003	308487
2004	353807
2005	476264
2006	573178

in forthcoming posts we're going to analise the coverage in PATSTAT against such figures

References

EPO international relation： http://www.epo.org/about-us/office/international-relations/projects/china.html
EPO and SIPO move ahead to secure IPR: http://www.epo.org/topics/news/2008/20080606.html

European patents and patent applications - statistics 2008: http://www.epo.org/about-us/office/statistics.html
Patent Information News 2/2009: http://www.epo.org/about-us/publications/patent-information/news/2009.html

Thanks for help in collecting the information to Kun Fu 

Origin of citations in European Patent Office

Whenever you'd need to understand who introduces a citation to a patent in PATSTAT, you may look at  table TLS212_citation, where you may find the field CITN_ORIGIN.

The content of such field may be any of these


0 - SEA- citations introduced during search
1 - APP- citations introduced by the applicant
2 - EXA- citations introduced during examination
3 - OPP - citations introduced during opposition
4 - 115- citations introduced according to Art 115 EPC
5,6 not public



Let's see the content of this field in EPO patents (count made on PATSTAT 10/2009 grouping by publication numbers)

ORIGIN        COUNT OF CITATIONS

0                    22985072

1                     6175203

2                      178744

3                       15839

4                       13533

While in in USPTO patents..
(count made with same criteria as above)

ORIGIN        COUNT OF CITATIONS
0              37425072 
1              15534910

The higher number of applicant introduced citations is due to the "duty of candor" rule which imposes the disclosure of all prior art.

Patstat and patent families

As wikipedia says, a patent family is "a set of patents taken in various countries to protect a single invention". Another way to say it is to call the patents belonging to the same family as 'equivalents'.

Infact when using ESPACENET and searching FI patent EP100000 you will see on rightmost side of your browser a list of "also published as" (1) that can be listed more in detail by going on "View INPADOC patent family" (2).


But we must be careful cause with patents, as in real  life, is very hard to give a unique definiton of what a family is...

In patstat from april 2009 two new tables have been introduced in order to help users to build a table of equivalents: tls218_DOCDB_FAM and tls219_INPADOC_FAM.

the first, tls218_DOCDB_FAM (SIMPLE FAMILY) gives the same family id to applications claiming exactly the same prior applications as priorities (these can be Paris Convention priorities or just technical relation priorities).
As PATSTAT documentation says: "The EPO reserve the right to classify an application into a particular simple family irrespective of this general rule" This is done by creating artificial priorities for an application to force it to match the priorities of a family.
The simple family is also at times used to attribute automatically the same IPC classification symbols and other attributes to corresponding applications.

The latter, tls219_INPADOC_FAM, (extended priority family) was developed by the INPADOC organisation then integrated by EPO.
In this case the linkage among applications can come from connections in tables TLS204_appln_prior (PARIS convention priorities) , TLS205_TECH_REL (patents which have been technically linked by patent examiners on the basis of similar content) and table TLS216_appln_contn (continuations, divisions etc).
The artificial PATSTAT applications due to priorities which have no entry in DOCDB are also included in this family.
The artificial PATSTAT applications due to unknown cited publications are included in this family table , but they all appear as a family with 1 member only.

Maybe an example can make things clearer: if we consider the group of applications D1..D5

Document D1    Priority P1

Document D2    Priority P1    Priority P2

Document D3    Priority P1    Priority P2

Document D4                   Priority P2    Priority P3

Document D5                                  Priority P3

While for INPADOC all documents will belong to one family only, for EPODOC we will have

FAMILY1: D1
FAMILY2: D2, D3
FAMILY3: D4
FAMILY4: D5

Or if you want a real case, I selected to which DOCDB and INPADOC family was belonging the application with patstat appln_id = 1; the difference is very evident.


DOCDB family           

APPLN_ID     DOCDB_FAMILY_ID    appl_auth    APPLN_NR

1            25590760            'AL'     '        9600001'

889876       25590760            'AT'     '       96931674'

1806521      25590760            'AU'     '        7081996'

18755226     25590760            'ES'     '       96931674'

               
INPADOC               

APPLN_ID     INPADOC_FAMILY_ID    appl_auth    APPLN_NR

1            3960163              'AL'     '        9600001'

889876       3960163              'AT'     '       96931674'

1806521      3960163              'AU'     '        7081996'

14573559     3960163              'DE'     '       69602451'

14573560     3960163              'DE'     '       69602451'

17633931     3960163              'EP'     '       96931674'

18755226     3960163              'ES'     '       96931674'

24291685     3960163              'GR'     '       99401928'

47191406     3960163              'US'     '        2963998'

57000038     3960163              'AL'     '           4195'

59131347     3960163              'EP'     '        9503551'

Limits of DOCDB and INPADOC

We may say the DOCDB families are sometimes too restrictive, since they do not put divisionals, continuations etc. into the same family although they should. Also the same appln_id may occur in more than one DOCDB family.

The INPADOC on the other side shows highly related patent documents, even if they are not necessarily on the same aspect of the invention. The same application id only occurs in one family, but the family is typically very comprehensive.

Other definitions:

Dietmar Harhoff at LMU developed a definition of equivalents allocating patents into one group/family of equivalents if they have the same priorities, and then it aggregates those groups across which members occur more than once.

Also Derwent developed a definition of patent family and here you can find some details.

(Thanks to Fabio Montobbio and Raffaele Conti for helping me in information collection for this post)

what is PATSTAT

Ok
now I realized I started this blog assuming everybody knows what PATSTAT is.
This is unfair, for many years I was unaware of PATSTAT and lived very happy...

Anyway: PATSTAT stands for EPO Worldwide Patent Statistical Database: created by EPO for use by government/intergovernmental organizations and academic institutions.

Contains a snapshot of the EPO master documentation database (DOCDB) which contains data of about 90 national and international patent offices with different degree of coverage.

Data include bibliographic data, citations and family links. This database is designed to be used for statistical research and requires the data to be loaded in the customer's own database (basically data are the same of DocDB but some fields have been added to make easier to retrieve statistical data like extra address information extracted from US and EP registers and standardized names).

Detailed coverage information: 
http://documents.epo.org/projects/babylon/eponet.nsf/0/2464E1CD907399E0C12572D50031B5DD/$File/global_patent_data_coverage_0709.pdf

http://documents.epo.org/projects/babylon/rawdata.nsf/0/74B501F17B1004BAC125765D004AA5B9/$File/PFS_0944.xls

Patent information received at EPO from national patent offices, are made available.  For many countries  data are received on a weekly basis, for other countries it is delayed.  The above PDF document sheds some light on this delays.

The database is constructed with a relational structure, containing 20 different tables. The compressed raw data is around 10 Gb (distributed in 3 DVDs) and once loaded into a database it can rise up to 100Gb (indexes included).

Tweaking MySql for Patstat

When dealing with Patstat under Mysql you should expect some slowdown to queries and table accessions due to the huge number of records of some tables.

So someway we should be more careful in the setup of Mysql DB engine and in table design ...

Let's start from one MYsql environment setup that is Buffers' size.

From Mysql administrator choose STARTUP VARIABLES, then in GENERAL PARAMETERS you will find the MEMORY USAGE slot.
(Or if you preferr edit the file MY.INI under programs\MYSQL\MySql server 5.1\)

Buffer size helps to execute operations in memory instead of using disk; performance may differ of 100-1.000 times depending from OS, table structure and so on.

On the other hands increasing buffer size will decrease preformance for other concurrent applications (If you, however, make this too big your system may start to page and become extremely slow....)

KEY buffer of 512 Mb and Sort Buffer of 1 Mb may be a good compromise (but I use a double amount for both when building some heavy tables like citations).

Also monitorying memory health status (MYSQL ADMINISTRATOR --> HEALTH --> MEMORY HEALTH)  may be a good issue for more precise setting of buffers.

Some more issues I'll discuss more in details in some next posts are about type of DB engine (InnoDB vs MyIsam) and type of indexes (HASH vs BTREE).

One last thing: when making some joins where one of the tables is very small, it may be helpful to load it into memory:

CREATE TABLE new_table ENGINE=MEMORY AS SELECT * FROM small_table;
Then recreate the index (using HASH) and make the join...

breaking my disk

ok
this week special is about bad luck!
what if my file system would corrupt and I forgot (or even worse did not schedule) a back up of my beloved data?
A good tool in such a case may be Testdisk, freeware able to recover lost partitions or file systems.
After downloading and running the program, by choosing Analyze you may select the partition no more recognized by the file system; then with the commands Proceed, Intel, Advanced, Boot and BackUp BS you will be able to recover the backup boot sector.
If also the Backup BS is corrupted you may try the command Rebuild Bs.
If also this case fails, well this other site offers a good choice of talismans against bad luck!

Other uses of TestDisk are described at this link: http://www.cgsecurity.org/wiki/Data_Recovery_Examples

Losing my password

Raise your hand if it never happened to you to forget a password.
If this password is the one of your PC Admin or of your user this is going to become a big problem...
There is also a chance that, if your machine has no UPS, a power cut may lock you user.

If you do not want to 'refresh' your PC/server by reinstalling the OS a solution may be Offline NT Password & Registry Editor, a tool that may be downloaded from the site http://pogostick.net/~pnh/ntpasswd/.
There you can download the image of a bootable cd that, inserted in the machine where you lost the password / have a user locked, allows you to reset the password or unlock the user.
Obviously it works only on local accounts; no network accounts or active directories.

A piece to have in house, hopefully without using it.