本文将带您了解关于SQL插入一行还是多行数据?的新内容,另外,我们还将为您提供关于c#–SQL插入一行或多行数据?、Excel表格中每行数据下面插入一行的方法、Java-向大查询重复插入一行数据问题、
本文将带您了解关于SQL插入一行还是多行数据?的新内容,另外,我们还将为您提供关于c# – SQL插入一行或多行数据?、Excel表格中每行数据下面插入一行的方法、Java - 向大查询重复插入一行数据问题、mysql 一次插入多行数据,获取的 insert_id 是第一行的 id 还是最后一行的 id?的实用信息。
本文目录一览:- SQL插入一行还是多行数据?
- c# – SQL插入一行或多行数据?
- Excel表格中每行数据下面插入一行的方法
- Java - 向大查询重复插入一行数据问题
- mysql 一次插入多行数据,获取的 insert_id 是第一行的 id 还是最后一行的 id?
SQL插入一行还是多行数据?
我正在一个控制台应用程序上将数据插入MS SQL Server 2005数据库。我有一个要插入的对象列表。这里我以Employee类为例:
List<Employee> employees;
我能做的就是像这样在一个时间插入一个对象:
foreach (Employee item in employees)
{
string sql = @"INSERT INTO Mytable (id,name,salary)
values ('@id','@name','@salary')";
// replace @par with values
cmd.CommandText = sql; // cmd is IDbCommand
cmd.ExecuteNonQuery();
}
或者,我可以像这样构建一个不插入查询:
string sql = @"INSERT INTO MyTable (id,salary) ";
int count = employees.Count;
int index = 0;
foreach (Employee item in employees)
{
sql = sql + string.format(
"SELECT {0},'{1}',{2} ",item.ID,item.Name,item.Salary);
if ( index != (count-1) )
sql = sql + " UNION ALL ";
index++
}
cmd.CommandType = sql;
cmd.ExecuteNonQuery();
我猜后一种情况将立即插入数据行。但是,如果我有几千个数据,SQL查询字符串是否有限制?
就性能而言,我不确定一次插入多行是否胜过一次插入一行数据?
有什么更好的建议吗?
c# – SQL插入一行或多行数据?
List<Employee> employees;
我可以做的是在时间上插入一个对象:
foreach (Employee item in employees) { string sql = @"INSERT INTO Mytable (id,name,salary) values ('@id','@name','@salary')"; // replace @par with values cmd.CommandText = sql; // cmd is IDbCommand cmd.ExecuteNonQuery(); }
或者我可以建立一个这样的balk插入查询:
string sql = @"INSERT INTO MyTable (id,salary) "; int count = employees.Count; int index = 0; foreach (Employee item in employees) { sql = sql + string.format( "SELECT {0},'{1}',{2} ",item.ID,item.Name,item.Salary); if ( index != (count-1) ) sql = sql + " UNION ALL "; index++ } cmd.CommandType = sql; cmd.ExecuteNonQuery();
我想后一种情况会一次性插入数据行.但是,如果我有
几个ks的数据,SQL查询字符串有什么限制吗?
我不知道如果一个插入多行比一行插入数据更好,在性能方面?
有什么建议要做的更好吗?
解决方法
>你的第二个例子有问题.你正在做sql = sql等.这将导致为循环的每个迭代创建一个新的字符串对象. (查看StringBuilder类).从技术上讲,您将在第一个实例中创建一个新的字符串对象,但不同之处在于它不必将以前的字符串选项中的所有信息复制.
>您设置的方式,sql Server将不得不潜在地评估一个大量的查询,当你终于发送它,这绝对需要一些时间来确定应该做什么.我应该说,这取决于你需要做的插件的数量.如果n小,你可能会很好,但是随着它的增长,你的问题只会变得更糟.
由于sql Server处理批量事务的方式,批量插入比单个插入更快.如果要从C#中插入数据,您应该采取第一种方法,并将每500个插入的内容包装在一个事务中并提交,然后执行下一个500等等.这也有一个优点,如果一个批次失败,你可以陷阱那些,弄清楚出了什么问题,重新插入那些.还有其他方法可以做到这一点,但这肯定会比提供的两个例子有所改进.
var iCounter = 0; foreach (Employee item in employees) { if (iCounter == 0) { cmd.BeginTransaction; } string sql = @"INSERT INTO Mytable (id,'@salary')"; // replace @par with values cmd.CommandText = sql; // cmd is IDbCommand cmd.ExecuteNonQuery(); iCounter ++; if(iCounter >= 500) { cmd.CommitTransaction; iCounter = 0; } } if(iCounter > 0) cmd.CommitTransaction;
Excel表格中每行数据下面插入一行的方法
Excel表格中每行数据下面插入一行的方法
方法一:
根据定位(快捷键F5),给空值插入行。
方法二:根据序号排序的方法。
Java - 向大查询重复插入一行数据问题
如何解决Java - 向大查询重复插入一行数据问题?
我正在创建一个会被第三方攻击的系统 我预计每分钟 25,000 次请求将有大约 1000 万次点击
这是我的代码
public class MyCustomClass extends HttpServlet {
private static final Logger LOGGER = LoggerFactory.getLogger(MyCustomClass.class);
public MyCustomClass() throws IOException,InterruptedException {
}
@Override
protected void doPost(HttpServletRequest req,HttpServletResponse resp) throws servletexception,IOException {
String body = req.getReader().lines().collect(Collectors.joining(System.lineseparator()));
ObjectMapper mapper = new ObjectMapper();
ThirdPartySpec spec = mapper.readValue(body,ThirdPartySpec.class);
insertDetail(spec);
resp.setContentType("application/json");
resp.setStatus(200);
}
private boolean insertDetail(ThirdPartySpec spec) {
try {
Map<String,Object> rowContent = new HashMap<>();
rowContent.put("field1",spec.getData1());
rowContent.put("field2",spec.getData2());
insertToBigQuery(bigquery,"myprojectid","mydatasetid",rowContent);
} catch (Exception e) {
LOGGER.error("error aa " + new Gson().toJson(spec),e);
return false;
}
return true;
}
private void insertToBigQuery(BigQuery bigQuery,String datasetName,String tableName,Map<String,Object> rowContent) throws Exception {
TableId tableId = TableId.of(datasetName,tableName);
InsertAllResponse response =
bigQuery.insertAll(
InsertAllRequest.newBuilder(tableId)
.addRow(rowContent)
.build());
if (response.hasErrors()) {
for (Map.Entry<Long,List<BigQueryError>> entry : response.getInsertErrors().entrySet()) {
throw new Exception(entry.getValue().toString());
}
}
}
我当前的配置
<instance-class>F4</instance-class>
<automatic-scaling>
<target-cpu-utilization>0.8</target-cpu-utilization>
<max-instances>10</max-instances>
<min-instances>1</min-instances>
<max-concurrent-requests>100</max-concurrent-requests>
</automatic-scaling>
我遇到了一些错误:
com.traveloka.accessor.mycustomclass insertDetail: error aa {"UserId":"123","name":"Superman"}
com.google.cloud.bigquery.BigQueryException: broken pipe (Write Failed)
at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.translate(HttpBigQueryRpc.java:115)
at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.insertAll(HttpBigQueryRpc.java:494)
at com.google.cloud.bigquery.BigQueryImpl.insertAll(BigQueryImpl.java:1036)
Caused by: java.net.socketException: broken pipe (Write Failed)
at java.net.socketoutputStream.socketWrite0(Native Method)
at java.net.socketoutputStream.socketWrite(SocketoutputStream.java:111)
at java.net.socketoutputStream.write(SocketoutputStream.java:155)
com.traveloka.accessor.mycustomclass insertDetail: error aa {"UserId":"123","name":"Superman"}
com.google.cloud.bigquery.BigQueryException: Remote host closed connection during handshake
at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.translate(HttpBigQueryRpc.java:115)
at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.insertAll(HttpBigQueryRpc.java:494)
at com.google.cloud.bigquery.BigQueryImpl.insertAll(BigQueryImpl.java:1036)
Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1004)
at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1783)
at sun.security.ssl.HandshakeOutStream.flush(HandshakeOutStream.java:124)
at sun.security.ssl.Handshaker.sendChangeCipherSpec(Handshaker.java:1156)
com.traveloka.accessor.mycustomclass insertDetail: error aa {"UserId":"123","name":"Superman"}
com.google.cloud.bigquery.BigQueryException: Read timed out
at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.translate(HttpBigQueryRpc.java:115)
at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.insertAll(HttpBigQueryRpc.java:494)
at com.google.cloud.bigquery.BigQueryImpl.insertAll(BigQueryImpl.java:1036)
Caused by: java.net.socketTimeoutException: Read timed out
at java.net.socketInputStream.socketRead0(Native Method)
at java.net.socketInputStream.socketRead(SocketInputStream.java:116)
at java.net.socketInputStream.read(SocketInputStream.java:171)
com.traveloka.accessor.mycustomclass insertDetail: error aa {"UserId":"123","name":"Superman"}
com.google.cloud.bigquery.BigQueryException: 502 Bad Gateway
POST https://www.googleapis.com/bigquery/v2/projects/myprojectname/datasets/mydatasetname/tables/mytablename/insertAll?prettyPrint=false
<!DOCTYPE html>
<html lang=en>
<Meta charset=utf-8>
<Meta name=viewport content="initial-scale=1,minimum-scale=1,width=device-width">
<title>Error 502 (Server Error)!!1</title>
<style>
*{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}
</style>
<a href=//www.google.com/><span id=logo aria-label=Google></span></a>
<p><b>502.</b> <ins>That’s an error.</ins>
<p>The server encountered a temporary error and Could not complete your request.<p>Please try again in 30 seconds. <ins>That’s all we kNow.</ins>
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 502 Bad Gateway
POST https://www.googleapis.com/bigquery/v2/projects/myprojectname/datasets/mydatasetname/tables/mytablename/insertAll?prettyPrint=false
Request was aborted after waiting too long to attempt to service your request.
我已经做了一些研究,但仍然没有线索
是否有任何修复方法或其他方法可用于在 bigquery 上存储数据?
仅供参考:
- 数据是从第三方每行发送的(这是最终的,不能更改)
解决方法
暂无找到可以解决该程序问题的有效方法,小编努力寻找整理中!
如果你已经找到好的解决方法,欢迎将解决方案带上本链接一起发送给小编。
小编邮箱:dio#foxmail.com (将#修改为@)
mysql 一次插入多行数据,获取的 insert_id 是第一行的 id 还是最后一行的 id?
mysql server 版本:8.0.31
Q: 对于 mysql 协议,insert 的返回值里面包含什么信息?是否包含 insert_id?
A: 通过 wireshark 抓包,结果如下:
执行 sql:
INSERT IGNORE INTO `user` (`name`, `age`) VALUES (''1'', 25), (''2'', 25), (''2'', 25);
mysql server 返回的内容如下:
Q:mysql 一次插入多行数据,获取的 insert_id 是第一行的 id 还是最后一行的 id?
A: 从上图可知,是第一个!mysql 的开发人员真是一群笨蛋!
测试代码如下:
from loguru import logger
from peewee import *
import settings
from playhouse.shortcuts import ReconnectMixin
from pymysql.cursors import Cursor
host = settings.MYSQL_CONFIG.host
port = settings.MYSQL_CONFIG.port
username = settings.MYSQL_CONFIG.username
password = settings.MYSQL_CONFIG.password
database_name = settings.MYSQL_CONFIG.database_name
class ReconnectMySQLDatabase(ReconnectMixin, MySQLDatabase):
pass
db = ReconnectMySQLDatabase(
database=database_name,
host=host,
port=port,
user=username,
password=password,
charset=''utf8mb4''
)
class User(Model):
name = CharField(unique=True)
age = IntegerField()
created_at = DateTimeField(
null=False,
constraints=[SQL(''DEFAULT CURRENT_TIMESTAMP'')],
help_text=''使用数据库时间''
)
updated_at = DateTimeField(
null=False,
constraints=[
SQL(''DEFAULT CURRENT_TIMESTAMP''),
SQL(''ON UPDATE CURRENT_TIMESTAMP''),
]
)
class Meta:
database = db
table_name = ''user''
model_set = [User]
db.drop_tables(model_set)
db.create_tables(model_set)
with db.cursor() as cursor:
cursor: Cursor
sql = """
INSERT IGNORE INTO `user` (`name`, `age`) VALUES (''1'', 25), (''2'', 25), (''2'', 25);
""".strip()
print(''affected_rows: '', cursor.execute(sql))
print(''lastrowid: '',cursor.lastrowid)
cursor.connection.commit() # 这里必须要提交,不然所有的查询都会处于一个事务中
关于SQL插入一行还是多行数据?的问题就给大家分享到这里,感谢你花时间阅读本站内容,更多关于c# – SQL插入一行或多行数据?、Excel表格中每行数据下面插入一行的方法、Java - 向大查询重复插入一行数据问题、mysql 一次插入多行数据,获取的 insert_id 是第一行的 id 还是最后一行的 id?等相关知识的信息别忘了在本站进行查找喔。
本文标签: