File Content Extraction 26.3
Filter SDK C++ Programming Guide

# Source Code Identification

Source code identification attempts to identify files that contain a particular programming language as a specific format.

When you do not enable source code identification, files that contain source code are often identified as ASCII text files, which means the application treats them in the same way as ordinary text. However, in many instances, it can be useful to route these files elsewhere or filter them out. For example, indexing source code into a Knowledge Discovery index has minimal value and might bloat the engine with terms that are of no use in retrieval.

To detect the format of a file, File Content Extraction uses the content of the file, rather than the file extension. While some file formats, such as PDF, have distinctive signatures that make them easy to identify, source code files tend to be less distinctive. For example, a fragment of JavaScript might look very similar to a fragment of another language such as Java, C#, C, or C++, or even plain text. File Content Extraction uses an analysis of the text that appears in the file to calculate the likelihood of it being in a particular programming language.

You can set source code identification to different levels.

Option | Description
--- | ---
`KVSOURCECODE_OFF` | Do not enable source code identification.
`KVSOURCECODE_CORE` | Fast and accurate detection of 28 of the most common source code formats, including C, C++, C#, CSS, Java, JavaScript, Lua, PHP, Perl, Powershell, Python, Rust, Shell, SQL, and TypeScript. The mode `KVSOURCECODE_CORE` is significantly faster than `ENABLED` and `EXTENDED`, and offers much better precision and recall. In this mode, File Content Extraction can also detect files as SourceCode_Fmt, which indicates that the file is likely to contain source code but the language could not be confidently determined. List of supported formats CPlusPlus_Fmt C_Fmt Clojure_Fmt Csharp_Fmt Css_Fmt Dart_Fmt Elixir_Fmt Fortran_Fmt Go_Fmt Haskell_Fmt Java_Fmt Javascript_Fmt Kotlin_Fmt Lua_Fmt Matlab_Fmt PHP_Fmt Perl_Fmt Powershell_Fmt Python_Fmt R_Fmt Ruby_Fmt Rust_Fmt SQL_Fmt Scala_Fmt Shell_Fmt Swift_Fmt Tex_Fmt TypeScript_Fmt
`KVSOURCECODE_ENABLED` | Enable source code identification for the most common source code formats. This option can detect formats 498-545, which would otherwise be detected as `ASCII_Text_Fmt`.
`KVSOURCECODE_EXTENDED` | Enable source code identification for all supported source code formats. This option might lead to false positives in some cases (for example, a C++ file might get identified as a rarer format). This option can detect formats 498-545, and 749-907, which would otherwise be detected as `ASCII_Text_Fmt`.

For the complete list of source code formats supported by these options, see Supported Formats.

TIP: Source code identification does not correspond exactly to the `adSOURCECODE` file class. Most files that are detected by source code identification have this file class, but there are exceptions. Also be aware that some formats in the `adSOURCECODE` class can be detected without enabling source code identification.

To configure source code identification

- In the C++ API, call `source_code_identification` on your `Configuration` object. For example:

      session.config().source_code_identification(keyview::v1::SourceCodeIdentificationLevel::Enabled);

- In `formats.ini`, set the following parameter to the appropriate level. (This is an alternative approach - you do not need to do this if you have configured this feature through the API).

      [Options]
      SourceCodeDetection=KVSOURCECODE_ENABLED

Copyright 1997-2026 Open Text
Version 26.3 | Last updated July 2026

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: File Content Extraction 26.3

Topic Title: Source Code Identification