Wednesday, September 15, 2010

Adding legal status to patents (inpadoc to mysql - part I)

In order to create a database containing data regarding legal issues of patents, epo provides a set of raw data called inpadoc database – legal status data (product 14.11) containing legal status data also known as PRS data (Patent Register Service) and includes records from over 40 international patent authorities.


The legal status of a patent or patent application refers to the entries and procedural steps occurring during the patent grant procedure and the subsequent life of a patent. These are normally published in the patent gazette of the patent-granting country or organisation concerned [from EPO website].


Legal status data are available in two formats: back file (from 1978 upto current year) and weekly updates (for current year). Following instruction refer to backlog file, but are applicable also for weekly update.

Inpadoc legal data come in two batches of files containing legal data (t12bfYYWW like where YY and WW are year and week of issue) and extended legal data (xlevYYWW) which translates the weekly bibliographic file (DOCDB) into legal events.

Data come in a XML format, splitted into several files due to the size of data.
Each file (example for T12BF) has a header

[iprevent cntevents=000656001 cntiprevents=000656003 date=20100129 record=START week=201004][/iprevent]

And records are structured like this

[iprevent cy=AT date=20100129 record=DATA status=C][l001ep]AT[/l001ep][l002ep]F[/l002ep][l003ep]168[/l003ep][l004ep]A [/l004ep][l005ep]PI[/l005ep][l007ep]19910315[/l007ep][l008ep]ELJ [/l008ep][l013ep]C[/l013ep][l017ep] AT 321347B[/l017ep][l018ep]20030101[/l018ep][/iprevent]

[NOTE: I changed < and > with [ and ] otherwise html interpreter would not display it on web page]

2010 backlog DVD contained, in T12BF group, 112 files and over 73M records, counting 24 Gb of unzipped data, while for XLEV was 155 files for 66.2 Gb of data.
If you want to create both tables, be sure to have 250 Gb of free space on disk.

In order to create a mysql table containing the relevant data the following steps can be followed:

1) file merge
After moving the file named T12BFYYMM___STAT (containing statistics about number of records for each file) files should be merged into one big file in order to facilitate the import process.
In order to facilitate the import step I suggest to also create a file with a record containing all the fields to be parsed, naming it @header.

Via DOS command line, move to the directory containing the files and run the following command

copy /b *.* patlegal.xml

This step may take 30 minutes.

2) MYSQL Import
The tool used for importing the file was NAVYCAT PREMIUM that has very powerful import features for mysql as well as other databases.
By rightclicking in the chosen library and selecting IMPORT WIZARD, you may chose XML import; be aware to chose IPREVENT as tag identifying table rows, when requested at step 3; for the rest you can just click on NEXT.

I suggest to create before via SQL an empty table having the correct field dimensions, so Navycat should only append the reocrds in the right fields.

In our case I created the table TEST.PATLEGAL.

This step may take 3 hrs.

[to be continued]

Thursday, September 9, 2010

code for IPC reclassification in 35 classes

In some previous posts I made available some code about ISI-OST-INPI IPC reclassification in 30 classes.

Recently, in order to have a better balance of the number of patents contained in the different classes, it has been proposed a 35 classes reclassification.

At this link can be consulted the ufficial document from Ulric Schmoch, published from WIPO.

So I made, in order to give am easier format, an excel file containing descriptions in one sheet and IPCs (in the second sheet); the file can be downloaded from this link.
Care the reclassifications sheet contains a colums IPCNOT showing that, if the patent contains such IPCs should be discarded from the class, regardless other IPCs should belong to it.

Differently from ISI-OST-INPI, this reclassification should be run on ALL IPCs (main and secundary).
As the author correctly adresses, this will lead to a 20% of patents belonging to 2 or more classes and for such cases only the main IPC (if available) should be used for reclassifications.
Have fun!!!

Tuesday, August 31, 2010

how to add NUTS to patent data in patstat

As cited in a previous post, when trying to implement a geographic reclassification of patents applicants and inventors, NUTS  classification provides a hierarchical system that lists, for EU, 97 regions at NUTS 1, 271 regions at NUTS 2 and 1303 regions at NUTS 3.


One road may be to use OECD REGPAT database, that covers patent applications filed to the EPO and PCT patents at international phase. Data mainly derives from PATSTAT [this means, if you use the same ediction of your patstat, you may link it via person_id]. The regional allocation is based on the latest Nomenclature of Territorial Units for Statistics (NUTS), 2006 for European regions, and on Territorial Level 3 for other countries; datasets are available for download to researchers: by sending an e-mail to sti.contact@oecd.org. Further methodological information are available on the OECD web site at http://www.oecd.org/sti/ipr-statistics.

In order to make a home-made linkage to NUTS3 may also be useful the link between postcodes and NUTS codes, that is freely available for following 20 countries at this eurostat web page:
http://epp.eurostat.ec.europa.eu/portal/page/portal/nuts_nomenclature/correspondence_tables/postcodes_and_nuts

Monday, July 19, 2010

PATSTAT: an assessment on inventors' addresses quality

When trying to consolidate data on inventors in PATSTAT, a part from name, we need some toponomastic data.
An innovator named "J. Smith" would be difficult to connect either with "John Smith" or with "James Smith", even if we have some further data like address, city...

So I investigated with some queries on patstat ediction 10/2009 in what percentuage the fields address, city, zip code and country are filled, by patent office.

I used tables TLS201_APPLN for linking application authority, and TLS206_ASCII (that is the ascii/parsed version of TLS206, included in patstat) for person ids and data.

For selecting distinct person id for inventors (A_I_FLAG = 'I') I used this SQL

Select
  T1.APPLN_AUTH, Count(Distinct t6.PERSON_ID)
From
  tls201_appln T1 Inner Join tls206_ascii t6 

  On t6.APPLN_ID = T1.APPLN_ID
Where   t6.A_I_FLAG = 'I';

Here below I'm pasting the  resulting table for the TOP 20 by inventor count, the full table can be downloaded @ this link.


APPLN_AUTH inventors no state no zip no country no address no city
US 5960856 86% 98% 21% 97% 25%
EP 3705123 100% 100% 0% 1% 1%
DE 2750079 100% 100% 33% 100% 100%
JP 1798271 100% 100% 98% 99% 100%
CN 1537587 100% 100% 2% 100% 100%
CA 1120490 100% 100% 45% 100% 100%
AU 1087573 100% 100% 98% 100% 100%
SU 968915 100% 100% 41% 100% 100%
AT 653048 100% 100% 29% 100% 100%
KR 637296 100% 100% 14% 100% 100%
FR 565254 100% 100% 98% 99% 100%
GB 531087 100% 100% 70% 65% 100%
RU 394691 100% 100% 29% 100% 100%
CH 338739 100% 100% 11% 100% 100%
BR 292047 100% 100% 89% 100% 100%
SE 256248 100% 100% 85% 98% 100%
FI 212722 100% 100% 11% 43% 100%
IT 192460 100% 100% 74% 100% 100%
ES 133471 100% 100% 17% 100% 100%
DD 129845 100% 100% 7% 97% 100%

It might be argued that such a count could have more sense if done by publication authority instead of application authority, since data are taken from search report and it can be noticed that FI data applied @ EPO and published @ WIPO/PCT differ in quality from those applied and published @ EPO.
Maybe it could be topic of a further post...

Wednesday, July 14, 2010

KITeS Patstat based reports

I've been vacating this blog for the majority of june because I implemented a set of reports based on Patstat EPO data.

At web address db.kites.unibocconi.it you will be able to find the following reports


Patent count by inventor country / priority year
Patent count by applicant country / priority year
Patent count by inventor region / priority year
Patent count by applicant region / priority year
Patent count by inventor nuts3 / priority year
Patent count by applicant nuts3 / priority year
Patent count by applicant name / priority year
Patent count by main IPC - first 4 digits
Patent count by main IPC class reclassified on OST30
Patent count by applicant, priority year, OST30 IPC reclassification
Patent count by applicant country, county, region, OST30 IPC reclassification, priority year
Citations count by applicant name, priority year
Citations count by applicant country, priority year
Citations count by inventor country, priority year
Copatenting by inventor country, priority year
Copatenting by applicant country, priority year
Applicants by IPC - first 4 digits
Inventors by IPC - first 4 digits

Please remember this is still a beta release; feel free to report me any problem or suggestion...

Eventually you will find some other reports password protected; such reports contain detail data and cannot be distributed since PATSTAT data are property of EPO.

Wednesday, July 7, 2010

Inventors data in patstat: Epo vs uspto

Patstat stores data about applicants and inventors inside a table with the prefix TLS206, indexed using a field named PERSON_ID that can be linked to the applications via table TLS207_PERS_APPLN linking each person_id to an application id (appln_id)Patstat DVD provides two versions of TLS206: the first TLS206_PERSON is a comma separated value file containing mainly the fields name, address and country code.
TLS206_ASCII instead contains the same information already parsed (and also something more, see below for fields list): name is splitted into last, first and middle name; address into street, city, state and zip code.

Data origins for person names and addresses:


1) EPO Register for EP patent applications; details are those that were the most recent in the EP Register at the time of extraction of the data.
2) OECD patents database for US data post 1976-01-01 up to and including November 15th 2005 for Published Grants.
3) PATSTAT weekly file extracts from USPTO website for Published Grants from November 22nd 2005 until today;  Published Applications  from September 29th 2005 to today inclusive.
4) Inventor & Applicant names for US PTO Published Applications from March 1st 2001 to September 22nd 2005 from DOCDB ,  data-format="docdba".
5) all other names  from DOCDB , data-format="docdba" (US data for names and addresses for patents published before 1976-01-01 is taken from the EPO's DOCDB Database)

TLS206_ASCII FIELD LIST                           

prof-last-name
prof-first-name
prof-middle-names
prof-street
prof-city
prof-state
prof-zip-code
 
TLS206_person field list                      

Person_id                         
Person_name                 
Person_address                             
Person_ctry_code         

Monday, May 31, 2010

patstat: counting applications by patent office

This is an answer to a very simple question: how many patent offices are listed in patstat and how many applications from each of them?
By aggregating the data we get the table I'm posting here below, containing 174 application authorities (data from 10/2009 patstat version). When considering instead only those who have more than 1000 applications, the figure decreases to 78 patent offices.
Some will notice that among the 174 the achronim WO (WIPO/PCT patent office) does not appear. This is because WO pubblications always result from an application to a local patent office (domestic or EP) where the PCT procedure is applied. The extent of this practice will be subject of one further post.



APPLN_AUTH count

1
AE 36
AF 1
AL 3
AM 147
AN 2
AP 4742
AR 79581
AT 1028408
AU 1553990
AZ 105
BA 344
BB 5
BD 9
BE 642989
BG 53602
BH 1
BI 9
BO 10
BR 496326
BS 2
BW 1
BX 105
BY 761
BZ 1
CA 2580140
CG 3
CH 1055204
CI 2
CL 4453
CM 7
CN 2803867
CO 222
CR 379
CS 166312
CU 2843
CY 2621
CZ 66921
DD 266209
DE 6792342
DK 429570
DM 12
DO 191
DZ 1547
EA 13114
EC 4949
EE 6251
EG 11450
EM 4254
EP 2388584
ER 6
ES 902689
FI 271371
FR 2917043
GA 2
GB 3319126
GC 420
GD 3
GE 186
GH 7
GI 5
GM 10
GN 8
GR 97444
GT 1301
HK 69450
HN 10
HR 11297
HT 7
HU 137061
IB 65369
ID 14768
IE 91248
IL 160970
IN 66238
IQ 14
IR 88
IS 7797
IT 708724
JM 6
JO 22
JP 16282860
KE 1392
KG 19
KP 60
KR 1659443
KZ 477
LB 109
LI 22
LK 152
LR 3
LS 3
LT 3651
LU 68491
LV 4835
LY 2
MA 10092
MC 2789
MD 4586
MG 1
MK 87
MN 246
MO 1
MR 1
MT 566
MU 7
MW 739
MX 162007
MY 11106
NE 3
NG 19
NI 207
NL 607873
NN 4
NO 226939
NP 2
NR 2
NZ 109638
OA 12934
OM 1
PA 2108
PE 431
PH 23261
PK 147
PL 233730
PT 81034
PY 12
RH 40
RO 61095
RS 73
RU 394084
SA 73
SB 1
SC 1
SD 67
SE 831377
SG 51063
SH 1
SI 17619
SK 23261
SL 2
SM 44
SN 10
SO 1
SR 3
ST 1
SU 1249050
SV 692
SY 44
SZ 4
TH 260
TJ 375
TM 2
TN 269
TR 42800
TT 52
TW 369739
TZ 1
UA 50213
US 11376401
UY 6573
UZ 53
VA 6
VE 100
VN 240
WO 5065
XH 1242
XP 16
YE 1
YU 33687
ZA 256542
ZM 2742
ZR 1
ZW 2909