英文:
unable to convert json to polars dataframe,'read an Array from a non-Array JSON''
问题
以下是您要翻译的内容：
我在将从API提供的数据反序列化后的JSON转换为Polars DataFrame方面遇到了困难。以下是我的结构和定义的函数：// 用于期权价格查询的结构#[derive(Serialize, Deserialize, Debug)]pub struct OptionResponse {pub tables: Vec<OptionTable>,}#[derive(Serialize, Deserialize, Debug)]pub struct OptionTable {pub table: OptionDataTable,}#[derive(Serialize, Deserialize, Debug)]pub struct OptionDataTable {pub volume: Vec<f64>,pub open: Vec<f64>,pub close: Vec<f64>,pub high: Vec<f64>,pub low: Vec<f64>,}pub async fn ifind_option_read(option_code: &str, access_token: &str, start_time_str: &str, end_time_str: &str) -> Result<(), Box<dyn std::error::Error>> {let url = "https://quantapi.51ifind.com/api/v1/high_frequency";// 创建高频价格请求let request_data = maplit::hashmap! {"codes" => option_code,"indicators" => "volume,open,close,high,low","starttime" => start_time_str,"endtime" => end_time_str,};let request_json = serde_json::to_string(&request_data)?;// 发送HTTP POST请求let client = reqwest::Client::new();let response = client.post(url).header(reqwest::header::CONTENT_TYPE, "application/json").header("access_token", access_token).body(request_json).send().await?;let response_json: OptionResponse = response.json().await?;let response_body = &response_json.tables[0].table;println!("{:?}", &response_body);// 将response_json转换为字节数组let json_bytes = serde_json::to_vec(&response_body)?;// 通过JsonReader的方法将响应读入dflet df = JsonReader::new(Cursor::new(json_bytes)).with_batch_size(1024).infer_schema_len(None).finish().unwrap();println!("{:?}", df);Ok(())}
返回的response_body看起来对我来说非常正常，类似于：
OptionDataTable { volume: [0.0, 0.0, 0.0], open: [1263.0, 1263.0, 1263.0], close: [1263.0, 1263.0, 1263.0], high: [1263.0, 1263.0, 1263.0], low: [1263.0, 1263.0, 1263.0] }
尽管如此，JsonReader无法将其读取为DataFrame。这让我感到困惑。这是JSON格式的问题吗？
<details><summary>英文:</summary>i am struggling with conversion from JSON(deserialised from API-provicded data) to polars dataframe.below are my structs and defined function:```rust// structs for option price query#[derive(Serialize, Deserialize, Debug)]pub struct OptionResponse {pub tables: Vec<OptionTable>,}#[derive(Serialize, Deserialize, Debug)]pub struct OptionTable {pub table: OptionDataTable,}#[derive(Serialize, Deserialize, Debug)]pub struct OptionDataTable {pub volume: Vec<f64>,pub open: Vec<f64>,pub close: Vec<f64>,pub high: Vec<f64>,pub low: Vec<f64>,}pub async fn ifind_option_read(option_code: &str, access_token: &str, start_time_str: &str, end_time_str: &str) -> Result<(), Box<dyn std::error::Error>> {let url = "https://quantapi.51ifind.com/api/v1/high_frequency";// create the high-frequency prices requestlet request_data = maplit::hashmap! {"codes" => option_code,"indicators" => "volume,open,close,high,low","starttime" => start_time_str,"endtime" => end_time_str,};let request_json = serde_json::to_string(&request_data)?;// send the HTTP POST requestlet client = reqwest::Client::new();let response = client.post(url).header(reqwest::header::CONTENT_TYPE, "application/json").header("access_token", access_token).body(request_json).send().await?;let response_json: OptionResponse = response.json().await?;let response_body = &response_json.tables[0].table;println!("{:?}", &response_body);// Convert response_json to byte arraylet json_bytes = serde_json::to_vec(&response_body)?;// read the response into df, per method of JsonReaderlet df = JsonReader::new(Cursor::new(json_bytes)).with_batch_size(1024).infer_schema_len(None).finish().unwrap();println!("{:?}", df);Ok(())}
the response_body returned looks very normal to me and looks like
OptionDataTable { volume: [0.0, 0.0, 0.0], open: [1263.0, 1263.0, 1263.0], close: [1263.0, 1263.0, 1263.0], high: [1263.0, 1263.0, 1263.0], low: [1263.0, 1263.0, 1263.0] }
neverthless the JsonReader cant read it as a df.
this is confusing. is this a json format problem?
答案1
得分: 0
以前在polars/polars-json/src/json/deserialize.rs中的deserialize函数使用了todo!宏，导致了这个错误。
pub fn deserialize(json: &BorrowedValue, data_type: DataType) -> Result<Box<dyn Array>, Error> {match json {BorrowedValue::Array(rows) => match data_type {DataType::LargeList(inner) => Ok(_deserialize(rows, inner.data_type)),_ => todo!("从非数组数据类型中读取数组"),},_ => todo!("从非数组JSON中读取数组"),}}
现在用Ok(_deserialize(&[json], data_type))替代了。
通过提交#207e0b9，现在一切都应该正常了。
英文:
previously the deserialize function in
polars/polars-json/src/json/deserialize.rs has a todo! macro causing this error.
pub fn deserialize(json: &BorrowedValue, data_type: DataType) -> Result<Box<dyn Array>, Error> {match json {BorrowedValue::Array(rows) => match data_type {DataType::LargeList(inner) => Ok(_deserialize(rows, inner.data_type)),_ => todo!("read an Array from a non-Array data type"),},_ => todo!("read an Array from a non-Array JSON"),}}
now it is replaced with Ok(_deserialize(&[json], data_type))
with commit #207e0b9 everything should be fine now.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论