> ## Documentation Index > Fetch the complete documentation index at: https://takeprofit.com/docs/llms.txt > Use this file to discover all available pages before exploring further. > Requirements for external CSV files in Indie: file structure, time formats, value parsing, hosting requirements, size limits and the error code reference. # CSV format and limits This page lists the requirements for CSV files used as external data sources, how the files are fetched, and what the error codes mean. ## File structure * UTF-8 encoding; a leading BOM is tolerated. * Comma (`,`) is the only supported delimiter — no semicolons, tabs or auto-detection. * Standard CSV quoting (RFC 4180): a field containing commas, quotes or newlines must be enclosed in double quotes. * Both LF and CRLF line endings work; blank lines are ignored. * A header row is required. Column order does not matter, unknown extra columns are ignored. ## The time column The time column (default name `time`, case-insensitive, configurable via `sources.Csv(url, time_column=...)`) accepts exactly three formats: | Format | Example | | ----------------------------- | --------------------------------------------------- | | Unix timestamp, seconds | `1704153600` | | Unix timestamp, milliseconds | `1704153600000` | | RFC 3339 / ISO 8601 date-time | `2024-01-02T00:00:00Z`, `2024-01-02T09:30:00+02:00` | Notes: * Seconds vs milliseconds are distinguished by magnitude: values of 1012 and above are read as milliseconds. * A bare date without a time part (`2024-01-02`) is **not** accepted. * Timezone offsets are normalized to UTC; timestamps without an explicit offset are not accepted. * Timestamps must be **strictly increasing**: duplicates and out-of-order rows are rejected, nothing is sorted or deduplicated for you. * The resolved date must fall between years 1900 and 2200. * Candle CSVs do not accept sub-second timestamps. Typed-data CSVs have millisecond resolution. ## Cell values | Field type | Accepted values | | ---------- | ------------------------------------------------------------------------------------------------------- | | `float` | Decimal numbers, scientific notation (`1e5`). `NaN`, `Inf`, underscores (`1_000`) and hex are rejected. | | `int` | Plain decimal integers within the 32-bit range. `1.0` is not a valid int. | | `bool` | `true`, `false`, `1`, `0` (case-insensitive). | | `str` | Any text, stored verbatim. | **Missing values.** The only way to express "no value" is an empty (or whitespace-only) cell, and it is only allowed in columns mapped to `Optional` fields. Literal markers like `NA`, `N/A`, `null` or `-` are not recognized — they are parse errors in numeric and bool columns. In a candle CSV only the `volume` cell may be empty (it reads as 0). **Validation is all-or-nothing.** A single bad cell — an unparseable timestamp, an empty required value, a malformed number — rejects the entire file. Rows are never silently skipped or filled with defaults. ## Hosting requirements * HTTPS only, with a valid certificate. Plain `http://` URLs are rejected. * The file must be publicly reachable: no authentication headers can be sent, and URLs with embedded credentials (`https://user:pass@...`) are rejected. If you need access control, use signed URLs (the query string is part of the URL and is kept out of server logs). * Up to 3 redirects are followed; every hop must also be HTTPS. * Addresses on private and internal networks are blocked. * gzip-compressed responses are supported; all size limits apply to the decompressed data. ## Fetching and caching * The file is downloaded when the indicator instance is created, with a 30-second fetch timeout. The dataset is a frozen snapshot: it is not refreshed while the indicator runs. * Downloads are cached on the server for about 10 minutes. Re-adding an indicator within that window may serve the previous content of the file. * Requests to the same URL are throttled; under heavy concurrent use a fetch can fail with `external_rate_limited` — retry by re-adding the indicator. ## Limits | Limit | Value | | --------------------------------------- | --------- | | Maximum file size (after decompression) | 64 MiB | | Maximum data rows | 1,000,000 | | Maximum columns per row | 4,096 | | Maximum size of one row | 1 MiB | | Maximum parsed data size (typed CSVs) | 48 MiB | | Maximum total size of string cells | 32 MiB | | Maximum schema leaf fields | 64 | | Fetch timeout | 30 s | ## Error reference When an external source fails, the indicator shows the error code followed by a short explanation, for example `external_invalid_number: high 5 is below low 7 (line 12)`. Errors found while reading the file name the CSV line they come from. The codes are: | Code | Meaning | What to check | | -------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | | `external_fetch_failed` | The file could not be downloaded: DNS failure, TLS error, timeout, non-success HTTP status, or a blocked address. | The URL is public, HTTPS, and responds within 30 s. | | `external_rate_limited` | Too many fetches of external data in a short period. | Retry by re-adding the indicator. | | `external_response_too_large` | The file exceeds the size limit. | Reduce the file below 64 MiB. | | `external_too_many_rows` | More than 1,000,000 data rows. | Reduce the number of rows. | | `external_too_many_columns` | A row exceeds 4,096 columns or 1 MiB. | Reduce the number of columns / row size. | | `external_malformed_csv` | Broken CSV structure, or the URL returned an HTML page instead of a CSV file. | The URL serves the raw file, not a viewer or login page; quoting follows RFC 4180. | | `external_empty` | No header row, no data rows, or (for candles) no rows in the requested chart range. | The file has a header and data covering the chart period. | | `external_missing_column` | A required column is absent from the header. | Header names match: candle columns are case-insensitive, schema fields are case-sensitive, nested fields use dots. | | `external_invalid_time` | A time cell could not be parsed, a candle timestamp has a sub-second part, or two increasing typed timestamps are closer than one millisecond. | Use Unix seconds/milliseconds or RFC 3339. | | `external_time_not_increasing` | Duplicate or out-of-order timestamps. | Sort rows by time and remove duplicates. | | `external_invalid_number` | A numeric cell could not be parsed, or candle prices are inconsistent (e.g. `high` below `low`, negative volume). | Numeric cells and OHLC relations. | | `external_invalid_bool` | A bool cell is not `true`/`false`/`1`/`0`. | Bool cell values. | | `external_int_out_of_range` | An int value does not fit into a 32-bit range. | Declare the field as `float`. | | `external_missing_value` | An empty cell in a column mapped to a non-`Optional` field. | Fill the value or declare the field `Optional`. | | `external_timeframe_mismatch` | Rows are more frequent than the `time_frame` declared at the call site. | The declared time frame matches the file cadence. | | `external_schema_invalid`, `external_schema_too_large` | The declared dataclass schema is invalid or too large. | Schema rules in [Typed external data](/docs/indie/External-data/Typed-external-data#declaring-a-schema). | | `external_payload_too_large`, `external_string_data_too_large` | The parsed data exceeds 48 MiB, or string cells exceed 32 MiB in total. | Shorten or drop long string columns; reduce the number of rows. | | `external_source_not_supported` | External data sources are currently disabled. | Try again later or contact support. | ## Errors when declaring the source The URL itself is validated when the indicator instance is created, before any download, so these errors appear without the file ever being fetched: | Code | Meaning | | ---------------------------- | ----------------------------------------------------------------------------------------------------- | | `invalid_url` | The URL cannot be parsed as `https://host/...`, or contains a control character or an embedded space. | | `scheme_not_https` | The URL scheme is not HTTPS. | | `userinfo_not_allowed` | The URL embeds credentials (`https://user:pass@...`). | | `ipv6_literal_not_supported` | The host is an IPv6 address literal. | | `non_ascii_host` | The host contains characters outside ASCII. | | `invalid_source_param` | The `time_column` argument is empty or contains control characters. |