JSON-RPC File Upload: Base64, Multipart, or Separate Flow?

For JSON-RPC file uploads, use a separate flow: have the client upload the file via a dedicated endpoint (e.g., HTTP POST with multipart/form-data) and then pass a reference (ID or URL) to the JSON-RPC method. This avoids the overhead and limitations of base64 encoding and keeps the JSON-RPC protocol clean and efficient. Below, we explore why this is the recommended approach, how each method works, and when you might still choose an alternative.

Background: JSON-RPC and File Transfer

JSON-RPC is a remote procedure call protocol that uses JSON for encoding. It is designed for method calls, not bulk data transfer. The protocol specifies that requests and responses are JSON objects, which are text-based and have no native binary support. When you need to send a file, you must either encode it as text (base64) or bypass the JSON-RPC channel entirely for the file payload. The choice affects performance, complexity, and interoperability.

Base64 Encoding: How It Works and Its Costs

Base64 is a binary-to-text encoding that represents binary data in an ASCII string format. It works by dividing the binary data into 24-bit groups and mapping each 6-bit chunk to a printable character. This increases the data size by about 33% (4 characters for every 3 bytes). For example, a 1 MB file becomes roughly 1.37 MB of base64 text.

In a JSON-RPC request, you would include the base64 string as a parameter, like:

{"jsonrpc": "2.0", "method": "uploadFile", "params": {"filename": "report.pdf", "data": "JVBERi0xLjQK..."}, "id": 1}

This is simple and works over any transport that supports JSON. However, the size overhead and the CPU cost of encoding/decoding can be significant for large files. Additionally, many JSON parsers have string length limits, and some servers reject requests larger than a few megabytes. The entire file must be loaded into memory as a string, which can be problematic for very large files.

Multipart/form-data: The Web Standard

Multipart/form-data is the standard encoding for HTML forms and is widely used for file uploads over HTTP. It sends the file as raw binary data, with a MIME boundary separating different parts. This is efficient and supports large files without the 33% overhead of base64. However, it is not JSON, so it cannot be used directly in a JSON-RPC request. To combine it with JSON-RPC, you would need to either:

• Send the file as a multipart request to a separate endpoint, and then call the JSON-RPC method with a reference to the uploaded file.
• Send the JSON-RPC request as a part of the multipart body, with the file as another part. This is possible but requires custom server-side parsing.

The first option is cleaner and more aligned with the JSON-RPC spec. The second option mixes concerns and is rarely used.

Separate Flow: Upload First, Then Reference

The recommended approach is to decouple file transfer from the JSON-RPC call. Here is a typical flow:

1. Client uploads the file to a dedicated endpoint (e.g., /upload) using multipart/form-data.
2. The server stores the file and returns a unique identifier (e.g., file ID or URL).
3. Client calls the JSON-RPC method, passing the file ID as a parameter.
4. The server retrieves the file using the ID and processes it.

This approach has several advantages:

• It keeps JSON-RPC messages small and focused on the procedure call.
• It leverages HTTP’s built-in file handling, which is optimized for binary data.
• It allows for progress tracking, resumable uploads, and parallel uploads.
• It reduces memory usage on both client and server.

The main downside is the need for an additional endpoint and a two-step process. However, this is a common pattern in modern APIs (e.g., uploading an image to a CDN, then referencing it in a post).

How to Choose: Factors to Consider

Your choice depends on the file size, the transport, and your infrastructure. Here are the key factors:

• File size: For small files (under 1 MB), base64 is acceptable. For larger files, the overhead and memory usage become problematic, making separate flow preferable.
• Transport: If you are using WebSockets or a custom TCP transport that only supports JSON, base64 is the only option. Over HTTP, multipart is natural.
• Interoperability: If your clients are not web-based (e.g., mobile apps), they may not support multipart easily. Base64 works everywhere JSON is supported.
• Server capabilities: Some servers have request size limits. If your server rejects large JSON payloads, you must use a separate flow.
• Need for progress: Base64 encoding happens all at once, so you cannot show upload progress. Multipart uploads can be streamed, allowing progress bars.

Common Problems and Misconceptions

One common misconception is that base64 is more secure because it is not binary. This is false; base64 is just an encoding, not encryption. Security depends on the transport (e.g., HTTPS).

Another issue is that some developers try to send files as JSON strings without base64, which breaks the JSON structure if the file contains quotes or control characters. Always encode binary data as base64 if you must include it in JSON.

Also, be aware of the 33% size increase and the fact that base64 strings are often cached or logged, potentially exposing sensitive data. If your file is confidential, consider using a separate flow with proper access controls on the file endpoint.

Alternative: Custom JSON-RPC Extension

Some implementations extend JSON-RPC to handle binary data by using a custom content type or a special parameter. For example, you could define a parameter that is a URL to a file, and the server fetches it. This is essentially the separate flow, but the upload step is replaced by a server-side fetch. This is useful when the file is already hosted elsewhere (e.g., a cloud storage URL). However, it introduces a dependency on the file being accessible, and the server must have network access to fetch it.

Frequently Asked Questions

Can I send a file as a JSON-RPC parameter without base64?

No, JSON is a text format, so binary data must be encoded. Base64 is the standard way to represent binary data in JSON. Unless you use a separate flow, you must encode the file.

What is the maximum file size I can send with base64 in JSON-RPC?

There is no fixed limit, but practical limits are imposed by the server’s request size limit and memory. Most web servers default to a limit of a few megabytes. For files larger than that, use a separate upload flow.

Does base64 increase the file size?

Yes, base64 increases the size by about 33%. For example, a 10 MB file becomes about 13.3 MB of base64 text.

Is multipart/form-data faster than base64?

Yes, because it sends raw binary data without encoding overhead. The difference is more noticeable with larger files.

Can I use WebSockets with JSON-RPC for file uploads?

Yes, but you must base64 encode the file since WebSockets typically send text frames. Some WebSocket libraries support binary frames, but JSON-RPC over WebSockets is text-based, so base64 is the only way to include the file in the same message.

What is the best practice for uploading large files with JSON-RPC?

The best practice is to use a separate flow: upload the file via a dedicated HTTP endpoint (multipart/form-data) and then call the JSON-RPC method with a file ID. This is scalable, efficient, and avoids the pitfalls of base64.

Related

• Shoot for Eggs: Beginner Core Loop Guide
• Ninja Foodi Power Blender CB402: Trade-Offs to Know
• UK Broadband vs Europe & Southeast Asia: Why It Lags